Hi;
I have been thinking of what else we can/should split out sometime soon to
CPAN and it has occurred to me that the interface of the templates could
probably be split out (and maybe the mailer too).
Both of these however contain LedgerSMB-specific assumptions and would need
to be rewritten to some extent to address these. However with the
templates in particular, removing these assumptions would help us get rid
of a lot of the CGI assumptions we are currently making. In particular, I
am thinking of the automatic output options which I think should be
abstracted away or even removed.
A major justification for this is that reporting endpoints for my LedgerSMB
API gateway are much harder to implement across a Dancer interface than
they should be, because we assume we are operating in a CGI environment.
I would like to do this for the following reasons:
1. I would really like to be able to use the reporting generation in areas
outside of a CGI web application, and
2. It would be nice to be able to open up additional formats to third
party developers saying "Put it on CPAN and tell people they can install it
like so."
I would like to suggest that over the next major version or two I work on
refactoring out or abstracting away the LedgerSMB-specific and CGI-specific
assumptions to the templating interface, provide a dynamic format
registration mechanism, and the like. And then work towards breaking these
off once this is complete.
Are there any thoughts on this?
--
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor
lock-in.
http://www.efficito.com/learn_more
With the idea to simplify our LedgerSMB::Template handler, by moving all
headers (but the content-type) from there to LedgerSMB::PSGI, I went to
task:
In the referenced issue, you write
>>>
>>> The template should be responsible *only* to render the content and
>>> provide a file mime type back with it. This means that the output of
>>> template->render() should ...
>>>
>>> Assuming that "template" there corresponds to my definition of
>>> "LedgerSMB Template Engine" (concluded from the fragment of the second
>>> sentence), I really wonder what value the template engine is providing: our
>>> template processor has a render method too; the *only* thing it lacks is
>>> the content type return value. The content type can be easily added based
>>> on the extension of the processed template in a generic handler elsewhere
>>> though.
>>>
>>
>> Right.
>>
>>>
>>> In the same paragraph, you continue
>>>
>>> The Template MUST NOT assume HTTP, PSGI, or any other particular
>>> input/output format.
>>>
>>> Could you have a look and https://github.com/ledgersmb/L
>>> edgerSMB/blob/master/lib/LedgerSMB/Template.pm#L540 ? I think that that
>>> achieves what you want.
>>>
>>
>> Yes, though more specifically, I think if we *do not use* PSGI or output
>> options we achieve something much better. What I want to achieve is a
>> clearer separation of responsibility and I think this has cascading
>> benefits through the rest of the application as well as making the logic
>> more general.
>>
>
> Ok. You're proposing the return value being a pair: the expanded template
> and the mime-type. The current implementation is a list of headers (usually
> just the mime-type), the expanded template and a success/fail indicator (a
> triplet).
>
> If we want to go with just the pair, how are we going to report the
> success/fail indicator? (In other words, how do we report error
> conditions?) I'll move the handling of the cache control headers currently
> in "render_to_psgi()" to LedgerSMB::PSGI. That's a positive outcome of this
> discussion for sure.
>
I have local work-in-progress. I've successfully moved the characterset
addition to (utf-8) to all "text/" mime-types as well as the headers
disabling caching when "back button disabling" is enabled, to
LedgerSMB::PSGI.
However, I'm trying to move the Content-Disposition headers to
LedgerSMB::PSGI, but that's not quite so easy: the disposition header uses
knowledge from the template (the type and name of the report). This
information is not available on the LedgerSMB::PSGI level. So, how do I
meaningfully name the downloaded file in this new world order?
Regards,
--
Bye,
Erik.
http://efficito.com -- Hosted accounting and ERP.
Robust and Flexible. No vendor lock-in.
As a couple notes, I would add the following expansions to this.
"Allow the widest possible version range on any module, disallowing
individual malfunctioning versions to further extend the range"
I think we are better off documenting bugs in misbehaving versions of
modules and perhaps offering a module check utility rather than disallowing
certain versions of modules. I say this for two reasons:
1. Managing this list is annoying and deciding when to block a specific
version of a module is going to be an annoying and political decision, and
2. A block on a module version will not prevent someone who has an older
version of the module from upgrading to a blocked version since it is an
install-time check and another piece of software could require a newer
version and thus break the install-time check.
So I would suggest deciding required versions by first-supported version
and up, and providing a tool or just documentation for checking for other
problems in the mean time.
Secondly I would like to suggest that a focus on moving things to CPAN and
breaking the application into components for better testability.
"Require as few a possible modules in the expanded dependency tree (prefer
modules as direct dependencies which are already depended on implicitly)"
Additionally I think that when we spin things off we should shoot for
modules which are simple in interface, within a year or two will likely be
pretty close to bug free and unlikely to require dramatic modifications,
and therefore keep spin-offs throttled so they do not cause undue chaos for
packagers.
" Not depend directly on modules which have overlapping functionalities"
Big caveat here is that some modules that are used *because* they are
overlapping might be in a different position. Moo/Moose is a good example
and both are widely used enough that there is no harm in using Moo instead
of Moose in spun-off modules. However we still have some old paradigms in
the code that need to be cleaned up and we should also focus on this.
"Not require modules which only provide nice-to-have functionality"
Also, while I am happy to see optional modules used to provide nice-to-have
functionality it is worth noting that there are two things that need to be
guarded against here. The first are extra dependencies. The second is
extra code complexity that comes with adding such options. I would say
that both need to be justified before we should allow them and where we do
allow them we should think about how to set up proper interfaces so
we don't have lots of conditional module handling everywhere.
"Group feature dependencies into their respective features as much as
possible (so as to not require them for a more basic installation)"
A corollary here might be that long-run a lot of the application today
should be relegated to optional features as we go forward.
If someone doesn't need inventory tracking, why install it? If one doesn't
need more than GL, customers, vendors, and basic reporting, why install
more than that?
Of course we aren't there yet but I would suggest that a forward-thinking
view of dependencies we might want to think about it not only regarding the
dependency problems we encounter with maintaining and working on the
current code but also in the question of how modularised and loosely
coupled we may want the code to be in the future.
Best Wishes,
Chris Travers
--
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor
lock-in.
http://www.efficito.com/learn_more
Hi
For various reasons I have started to package some other software for
Gentoo and decided to put together an overlay for LedgerSMB. I wanted to
share my experiences here and also looking at where the efforts I have seen
on Debian, etc where we can probably make some improvements.
First the good: Packaging the dependencies for LedgerSMB from CPAN was
super-easy and straight-forward. It is as simple as using g-cpan to
generate an initial ebuild, editing that to ensure that we aren't
over-promising, and then writing a metadata.xml. I spent around an hour
packaging all of our span dependencies. It was really, really easy.
As a side note, one of the impressions I get from Jame (please correct me
if I am wrong) is that one of the headaches here is legal: making sure we
have all the right copyright information on all the right documentation,
and this right now is a manual process. I will come back to that later.
Packaging LedgerSMB is, on the other hand, very different. On one hand,
these are not shared libraries so the build cannot go in vendor perl, and I
am trying to support running multiple versions in parallel. Managing the
direct dependency tree itself has been as much effort as packaging all of
the dependencies. My approach is basically to:
1. Create an ebuild
2. Create a makefile which will overwrite the project makefile
3. Clobber the existing makefile with the new one in the ebuild
4. Point it at a directory (/usr/share/ledgersmb-1.4 for example)
5. Set up Plack (probably Starman at first.
6. Use an OpenRC init script (similar to Sys-V) to start Starman with each
of the installed versions
Right now I am on stage 2. Since there is a presumption against bundling,
I will use any existing dojo ebuilds and this makes some things harder as
well. However already packaging LedgerSMB has proved much more effort than
packaging all the dependencies combined.
One of the reasons I would like to see more stuff on CPAN from us is it
would probably simplify this a lot. It's easier to have 2-3 optional
dependencies each with 4 dependencies than to have 8-12 optional
dependencies in 2-3 groups. Also, tracking which options have been enabled
poses some complexity. From a Gentoo packaging perspective, the more we
can put on CPAN the better. (Yes, g-cpan is not perfect, but it handles
90% of the cases nearly perfectly and once you get an ebuild, updating to a
new version is super-easy).
On to where I see some issues with the spinoff modules.
99% of the time I have been contacted by packagers (usually Jame) over
packaging questions it is the nuts and bolts of things like copyright
dates, changelog entries, etc. I would like to suggest that these can be
subject to automatic testing. I would like to suggest the following global
rules we can enforce by Travis or a git hook:
1. Copyright dates *MUST* include the current year for *all* modules
changed, checked on every commit to master. This is something maybe we
could enforce with a git hook.
2. Every tag MUST have a have an associated changelog entry. Again,
something we could enforce via a git hook.
There is one specific gentoo situation which is that source urls, being
what they are, do not support backpan and cpan transparently together.
Therefore I have to keep the overlay current regarding all currently
supported versions and change urls to backpan were needed. That's a minor
issue though.
On the whole I expect working Gentoo packages in not too much longer and
look forward to how we can help make packaging less work.
--
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor
lock-in.
http://www.efficito.com/learn_more
Hello.
Sorry to be away. Various infections and physical traumas have left me doing only the
basics for about seven weeks. That seems to be passing.
Re: 1.5.9
What are recommendations for customizing and adding tables (and all that follows)?
Tools? Naming conventions? etc.
For parts, adding about 25 fields and 4 constraining tables similar to salutations.
Lesser invasions to vendors and customer data.
Suggestions and references will also be appreciated.
Thanks,
Rob
Hi,
In a recent issue, Chris proposed to factor out part of the code base into
a stand-alone CPAN release (which would IMO mean into a more-or-less stand
alone project). The same happened some years ago when he factored out
DBObject into PGObject, so, we have precedence here. The other way around
(refactored our code to use releases from CPAN) has happened as well.
>From my perspective, the reasons to refactor our code to make use of
(pre-existing) CPAN code are clear:
* Using pre-existing code from CPAN outsources maintenance (to an external
maintainer)
* Opens up opportunities for wider use in our own code base, because
usually replaces specific with generic code
* Ensures better testing of code in a wide variety of settings (hopefully
with bugfixes)
* Improves code-sharing in the wider Perl community (hopefully free-ing up
resources to take Perl and its modules a step further)
To me, the other way around would roughly work the same way. Meaning, that
we should be thinking about placing our code on CPAN, if:
* Our code is sufficiently generic to solve problems for others
* Others can depend on us to do the maintenance of the code
* Our code comes with sufficient quality assurance measures to be helpful
to others
* Our code is sufficiently stable not to require large refactorings to the
projects (which would mean that people can't really depend on the code -
yet)
Especially the last point - I think - is tricky: when largish refactorings
are in order, the fact that part of the code being refactored is loaded
onto Travis CI through CPAN is a major limitation on the speed that can be
had during the refactoring. The recent PGObject 1.x->PGObject 2.x
refactoring showed that.
I'm wondering: Is there anything else to be considered? (Reasons to factor
out, problems to be met after having separated, etc...)
Regards,
--
Bye,
Erik.
http://efficito.com -- Hosted accounting and ERP.
Robust and Flexible. No vendor lock-in.
Hi,
Currently we have 30 repositories in the LedgerSMB github project.
over half of them are related to PGObject modules and are reasonably
named in the form of
PGObject-foo-bar
However, not all of our repositories are named so sensibly.
In particular we now have 4 "packaging" repositories
- pkg-ledgersmb
- pkg-ledgersmb-1.4
- pkg-ledgersmb-1.5
- lsmb-overlay
From the names, it's obscure at best which distro etc each of those
repositories is for, more confusing is the fact that the first 3 seem to
be duplicates of each other, excepting for the fact that they target
different releases of our code.
The first 3 are for debian packages, while the fourth is the new repo
Chris has added for gentoo packaging.
To help with "discoverability" and generally make it more obvious what a
repo is for, I'd propose a naming scheme along the following lines.
$purpose-$project-$detail
For true "project" repositories such as LedgerSMB, and PGObject projects
$purpose may not make sense, but for others it likely does.
Some examples
- LedgerSMB
- PGObject
- PGObject-Type-DateTime
- pkg-debian-LedgerSMB
- pkg-gentoo-LedgerSMB
- migration_test_data-LedgerSMB
- docker-LedgerSMB
- docker-LedgerSMB-selenium-grid
- OPOS-LedgerSMB-integration
- LedgerSMB-PHP-bindings
I think it's probably early enough in the lifecycle of most of the
repositories that could do with renaming to rename them now.
I say that as the ones that would change have little use outside of our
project controlled resources.
As for having the pkg-ledgersmb-1.x repositories as well as the
unversioned one, Unless Jame has a reason to do otherwise, I'd suggest
we simply use tags in the pkg-ledgersmb repo instead.
Consider the situation if we ended up with packaging repos for every
different distro, and also every release series we do in the future,
things would quickly become unmanageable
Sorry, forgot to send to the mailing list as well...
---------- Forwarded message ----------
From: Robert J. Clay <rjclay(a)gmail.com>
Date: Sat, Aug 26, 2017 at 2:30 PM
Subject: Re: [ledgersmb-devel] ledgersmb project owned repository naming
To: David Godfrey <info(a)sbts.com.au>
David,
On Sat, Aug 26, 2017 at 6:43 AM, David Godfrey <info(a)sbts.com.au> wrote:
> Currently we have 30 repositories in the LedgerSMB github project.
> over half of them are related to PGObject modules and are reasonably named
> in the form of
> PGObject-foo-bar
>
> However, not all of our repositories are named so sensibly.
>
> In particular we now have 4 "packaging" repositories
> ....
> - lsmb-overlay
>
> ...., while the fourth is the new repo Chris has added for gentoo packaging.
For an 'ebuild' for Gentoo, which is not quite the same thing...
(A "package" for gentoo looks to be just an application source
archive, which an 'ebuild' for that app uses to build and install the
application as necessary.)
> Some examples ....
> - pkg-gentoo-LedgerSMB
An "overlay" is not a package it's an ebuild and having the word
"overlay' in the repo name for them is very commonly related to
Gentoo. (Or so it seems to be judging from a search on GitHub...)
--
Robert J. Clay
rjclay(a)gmail.com
--
Robert J. Clay
rjclay(a)gmail.com
Hi,
Since we moved to being a single-page app on the main app, I'm looking to
improve clarity in the UI/ tree.
What's the problem? I see a number of things being unclear:
* Out of all the files stored in UI/, only these are actual "top level"
documents:
- UI/login.html
- UI/logout.html
- UI/main.html
- setup/*.html
* All files, also the bits that are "html fragments" these days, include a
full html start- and end-tags as well as a header definition (which are
being thrown away upon receipt!)
* There are non-HTML documents (ODS, XLS, CSV, ...) in that directory
(which suggests that it's not solely about UI, but also about
data-export/report generation)
Apart from the mix of HTML documents, the directory contains assets that
*are* being served: javascript/images/css/... .
Should we separate the HTML fragments in order to clarify the status as
being a "non-toplevel document"?
Another thing: I think we should separate out the templates which have not
*just* an HTML representation, meaning that they are not (primarily) UI
related elements.
Regards,
--
Bye,
Erik.
http://efficito.com -- Hosted accounting and ERP.
Robust and Flexible. No vendor lock-in.
Hi,
Sending this mail to the mailing list, because I think that's where we need
to set our policies.
I'm starting this topic because I feel like it's come up several times now
and I think it's time to set a policy, with documented reasons as to why
we're working the way we do.
First, let me re-iterate what I think are our shared goals for LedgerSMB
(as related to our choice of dependencies). We want:
LedgerSMB to be easy to install, meaning
* On as many platforms as possible
* Allowing people to use their distro's packages as much as possible
* Running first and foremost on Perl versions in most distros
* Allowing for a wide range of PostgreSQL versions
* Integrating (through PSGI/reverse-proxy) with people's favorate webserver
Translating the above into requirements for modules we depend on, I think
we want to:
* Use modules which have been around for a while (to increase chances of
being packaged)
* Allow the widest possible version range on any module, disallowing
individual malfunctioning versions to further extend the range
* Require as few a possible modules in the expanded dependency tree
(prefer modules as direct dependencies which are already depended on
implicitly)
* Not depend directly on modules which have overlapping functionalities
* Not require modules which only provide nice-to-have functionality
* Group feature dependencies into their respective features as much as
possible (so as to not require them for a more basic installation)
With respect to modules used in development, I'd like to be less strict,
but only marginally so: we want it to be "simple enough" to create a
development setup for LedgerSMB.
Comments?
Regards,
--
Bye,
Erik.
http://efficito.com -- Hosted accounting and ERP.
Robust and Flexible. No vendor lock-in.
