August 2017 - devel - lists.freelock.com

RFC: Formalize templates interface and split out onto CPAN
by Chris Travers 31 Aug '17

31 Aug '17

Hi; I have been thinking of what else we can/should split out sometime soon to CPAN and it has occurred to me that the interface of the templates could probably be split out (and maybe the mailer too). Both of these however contain LedgerSMB-specific assumptions and would need to be rewritten to some extent to address these. However with the templates in particular, removing these assumptions would help us get rid of a lot of the CGI assumptions we are currently making. In particular, I am thinking of the automatic output options which I think should be abstracted away or even removed. A major justification for this is that reporting endpoints for my LedgerSMB API gateway are much harder to implement across a Dancer interface than they should be, because we assume we are operating in a CGI environment. I would like to do this for the following reasons: 1. I would really like to be able to use the reporting generation in areas outside of a CGI web application, and 2. It would be nice to be able to open up additional formats to third party developers saying "Put it on CPAN and tell people they can install it like so." I would like to suggest that over the next major version or two I work on refactoring out or abstracting away the LedgerSMB-specific and CGI-specific assumptions to the templating interface, provide a dynamic format registration mechanism, and the like. And then work towards breaking these off once this is complete. Are there any thoughts on this? -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

3 9

Returning only content-type header from LedgerSMB::Template
by Erik Huelsmann 31 Aug '17

31 Aug '17

With the idea to simplify our LedgerSMB::Template handler, by moving all headers (but the content-type) from there to LedgerSMB::PSGI, I went to task: In the referenced issue, you write >>> >>> The template should be responsible *only* to render the content and >>> provide a file mime type back with it. This means that the output of >>> template->render() should ... >>> >>> Assuming that "template" there corresponds to my definition of >>> "LedgerSMB Template Engine" (concluded from the fragment of the second >>> sentence), I really wonder what value the template engine is providing: our >>> template processor has a render method too; the *only* thing it lacks is >>> the content type return value. The content type can be easily added based >>> on the extension of the processed template in a generic handler elsewhere >>> though. >>> >> >> Right. >> >>> >>> In the same paragraph, you continue >>> >>> The Template MUST NOT assume HTTP, PSGI, or any other particular >>> input/output format. >>> >>> Could you have a look and https://github.com/ledgersmb/L >>> edgerSMB/blob/master/lib/LedgerSMB/Template.pm#L540 ? I think that that >>> achieves what you want. >>> >> >> Yes, though more specifically, I think if we *do not use* PSGI or output >> options we achieve something much better. What I want to achieve is a >> clearer separation of responsibility and I think this has cascading >> benefits through the rest of the application as well as making the logic >> more general. >> > > Ok. You're proposing the return value being a pair: the expanded template > and the mime-type. The current implementation is a list of headers (usually > just the mime-type), the expanded template and a success/fail indicator (a > triplet). > > If we want to go with just the pair, how are we going to report the > success/fail indicator? (In other words, how do we report error > conditions?) I'll move the handling of the cache control headers currently > in "render_to_psgi()" to LedgerSMB::PSGI. That's a positive outcome of this > discussion for sure. > I have local work-in-progress. I've successfully moved the characterset addition to (utf-8) to all "text/" mime-types as well as the headers disabling caching when "back button disabling" is enabled, to LedgerSMB::PSGI. However, I'm trying to move the Content-Disposition headers to LedgerSMB::PSGI, but that's not quite so easy: the disposition header uses knowledge from the template (the type and name of the report). This information is not available on the LedgerSMB::PSGI level. So, how do I meaningfully name the downloaded file in this new world order? Regards, -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

2 1

Re: [ledgersmb-devel] Setting a policy for our dependencies
by Chris Travers 30 Aug '17

30 Aug '17

As a couple notes, I would add the following expansions to this. "Allow the widest possible version range on any module, disallowing individual malfunctioning versions to further extend the range" I think we are better off documenting bugs in misbehaving versions of modules and perhaps offering a module check utility rather than disallowing certain versions of modules. I say this for two reasons: 1. Managing this list is annoying and deciding when to block a specific version of a module is going to be an annoying and political decision, and 2. A block on a module version will not prevent someone who has an older version of the module from upgrading to a blocked version since it is an install-time check and another piece of software could require a newer version and thus break the install-time check. So I would suggest deciding required versions by first-supported version and up, and providing a tool or just documentation for checking for other problems in the mean time. Secondly I would like to suggest that a focus on moving things to CPAN and breaking the application into components for better testability. "Require as few a possible modules in the expanded dependency tree (prefer modules as direct dependencies which are already depended on implicitly)" Additionally I think that when we spin things off we should shoot for modules which are simple in interface, within a year or two will likely be pretty close to bug free and unlikely to require dramatic modifications, and therefore keep spin-offs throttled so they do not cause undue chaos for packagers. " Not depend directly on modules which have overlapping functionalities" Big caveat here is that some modules that are used *because* they are overlapping might be in a different position. Moo/Moose is a good example and both are widely used enough that there is no harm in using Moo instead of Moose in spun-off modules. However we still have some old paradigms in the code that need to be cleaned up and we should also focus on this. "Not require modules which only provide nice-to-have functionality" Also, while I am happy to see optional modules used to provide nice-to-have functionality it is worth noting that there are two things that need to be guarded against here. The first are extra dependencies. The second is extra code complexity that comes with adding such options. I would say that both need to be justified before we should allow them and where we do allow them we should think about how to set up proper interfaces so we don't have lots of conditional module handling everywhere. "Group feature dependencies into their respective features as much as possible (so as to not require them for a more basic installation)" A corollary here might be that long-run a lot of the application today should be relegated to optional features as we go forward. If someone doesn't need inventory tracking, why install it? If one doesn't need more than GL, customers, vendors, and basic reporting, why install more than that? Of course we aren't there yet but I would suggest that a forward-thinking view of dependencies we might want to think about it not only regarding the dependency problems we encounter with maintaining and working on the current code but also in the question of how modularised and loosely coupled we may want the code to be in the future. Best Wishes, Chris Travers -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

4 11

Sharing experiences packaging for Gentoo
by Chris Travers 29 Aug '17

29 Aug '17

Hi For various reasons I have started to package some other software for Gentoo and decided to put together an overlay for LedgerSMB. I wanted to share my experiences here and also looking at where the efforts I have seen on Debian, etc where we can probably make some improvements. First the good: Packaging the dependencies for LedgerSMB from CPAN was super-easy and straight-forward. It is as simple as using g-cpan to generate an initial ebuild, editing that to ensure that we aren't over-promising, and then writing a metadata.xml. I spent around an hour packaging all of our span dependencies. It was really, really easy. As a side note, one of the impressions I get from Jame (please correct me if I am wrong) is that one of the headaches here is legal: making sure we have all the right copyright information on all the right documentation, and this right now is a manual process. I will come back to that later. Packaging LedgerSMB is, on the other hand, very different. On one hand, these are not shared libraries so the build cannot go in vendor perl, and I am trying to support running multiple versions in parallel. Managing the direct dependency tree itself has been as much effort as packaging all of the dependencies. My approach is basically to: 1. Create an ebuild 2. Create a makefile which will overwrite the project makefile 3. Clobber the existing makefile with the new one in the ebuild 4. Point it at a directory (/usr/share/ledgersmb-1.4 for example) 5. Set up Plack (probably Starman at first. 6. Use an OpenRC init script (similar to Sys-V) to start Starman with each of the installed versions Right now I am on stage 2. Since there is a presumption against bundling, I will use any existing dojo ebuilds and this makes some things harder as well. However already packaging LedgerSMB has proved much more effort than packaging all the dependencies combined. One of the reasons I would like to see more stuff on CPAN from us is it would probably simplify this a lot. It's easier to have 2-3 optional dependencies each with 4 dependencies than to have 8-12 optional dependencies in 2-3 groups. Also, tracking which options have been enabled poses some complexity. From a Gentoo packaging perspective, the more we can put on CPAN the better. (Yes, g-cpan is not perfect, but it handles 90% of the cases nearly perfectly and once you get an ebuild, updating to a new version is super-easy). On to where I see some issues with the spinoff modules. 99% of the time I have been contacted by packagers (usually Jame) over packaging questions it is the nuts and bolts of things like copyright dates, changelog entries, etc. I would like to suggest that these can be subject to automatic testing. I would like to suggest the following global rules we can enforce by Travis or a git hook: 1. Copyright dates *MUST* include the current year for *all* modules changed, checked on every commit to master. This is something maybe we could enforce with a git hook. 2. Every tag MUST have a have an associated changelog entry. Again, something we could enforce via a git hook. There is one specific gentoo situation which is that source urls, being what they are, do not support backpan and cpan transparently together. Therefore I have to keep the overlay current regarding all currently supported versions and change urls to backpan were needed. That's a minor issue though. On the whole I expect working Gentoo packages in not too much longer and look forward to how we can help make packaging less work. -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

1 0

customizations
by R. Ransbottom 27 Aug '17

27 Aug '17

Hello. Sorry to be away. Various infections and physical traumas have left me doing only the basics for about seven weeks. That seems to be passing. Re: 1.5.9 What are recommendations for customizing and adding tables (and all that follows)? Tools? Naming conventions? etc. For parts, adding about 25 fields and 4 constraining tables similar to salutations. Lesser invasions to vendors and customer data. Suggestions and references will also be appreciated. Thanks, Rob

1 0

When to factor out code into a stand-alone CPAN release?
by Erik Huelsmann 27 Aug '17

27 Aug '17

Hi, In a recent issue, Chris proposed to factor out part of the code base into a stand-alone CPAN release (which would IMO mean into a more-or-less stand alone project). The same happened some years ago when he factored out DBObject into PGObject, so, we have precedence here. The other way around (refactored our code to use releases from CPAN) has happened as well. >From my perspective, the reasons to refactor our code to make use of (pre-existing) CPAN code are clear: * Using pre-existing code from CPAN outsources maintenance (to an external maintainer) * Opens up opportunities for wider use in our own code base, because usually replaces specific with generic code * Ensures better testing of code in a wide variety of settings (hopefully with bugfixes) * Improves code-sharing in the wider Perl community (hopefully free-ing up resources to take Perl and its modules a step further) To me, the other way around would roughly work the same way. Meaning, that we should be thinking about placing our code on CPAN, if: * Our code is sufficiently generic to solve problems for others * Others can depend on us to do the maintenance of the code * Our code comes with sufficient quality assurance measures to be helpful to others * Our code is sufficiently stable not to require large refactorings to the projects (which would mean that people can't really depend on the code - yet) Especially the last point - I think - is tricky: when largish refactorings are in order, the fact that part of the code being refactored is loaded onto Travis CI through CPAN is a major limitation on the speed that can be had during the refactoring. The recent PGObject 1.x->PGObject 2.x refactoring showed that. I'm wondering: Is there anything else to be considered? (Reasons to factor out, problems to be met after having separated, etc...) Regards, -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

2 2

ledgersmb project owned repository naming
by David G 26 Aug '17

26 Aug '17

Hi, Currently we have 30 repositories in the LedgerSMB github project. over half of them are related to PGObject modules and are reasonably named in the form of PGObject-foo-bar However, not all of our repositories are named so sensibly. In particular we now have 4 "packaging" repositories - pkg-ledgersmb - pkg-ledgersmb-1.4 - pkg-ledgersmb-1.5 - lsmb-overlay From the names, it's obscure at best which distro etc each of those repositories is for, more confusing is the fact that the first 3 seem to be duplicates of each other, excepting for the fact that they target different releases of our code. The first 3 are for debian packages, while the fourth is the new repo Chris has added for gentoo packaging. To help with "discoverability" and generally make it more obvious what a repo is for, I'd propose a naming scheme along the following lines. $purpose-$project-$detail For true "project" repositories such as LedgerSMB, and PGObject projects $purpose may not make sense, but for others it likely does. Some examples - LedgerSMB - PGObject - PGObject-Type-DateTime - pkg-debian-LedgerSMB - pkg-gentoo-LedgerSMB - migration_test_data-LedgerSMB - docker-LedgerSMB - docker-LedgerSMB-selenium-grid - OPOS-LedgerSMB-integration - LedgerSMB-PHP-bindings I think it's probably early enough in the lifecycle of most of the repositories that could do with renaming to rename them now. I say that as the ones that would change have little use outside of our project controlled resources. As for having the pkg-ledgersmb-1.x repositories as well as the unversioned one, Unless Jame has a reason to do otherwise, I'd suggest we simply use tags in the pkg-ledgersmb repo instead. Consider the situation if we ended up with packaging repos for every different distro, and also every release series we do in the future, things would quickly become unmanageable

2 2

Fwd: ledgersmb project owned repository naming
by Robert J. Clay 26 Aug '17

26 Aug '17

Sorry, forgot to send to the mailing list as well... ---------- Forwarded message ---------- From: Robert J. Clay <rjclay(a)gmail.com> Date: Sat, Aug 26, 2017 at 2:30 PM Subject: Re: [ledgersmb-devel] ledgersmb project owned repository naming To: David Godfrey <info(a)sbts.com.au> David, On Sat, Aug 26, 2017 at 6:43 AM, David Godfrey <info(a)sbts.com.au> wrote: > Currently we have 30 repositories in the LedgerSMB github project. > over half of them are related to PGObject modules and are reasonably named > in the form of > PGObject-foo-bar > > However, not all of our repositories are named so sensibly. > > In particular we now have 4 "packaging" repositories > .... > - lsmb-overlay > > ...., while the fourth is the new repo Chris has added for gentoo packaging. For an 'ebuild' for Gentoo, which is not quite the same thing... (A "package" for gentoo looks to be just an application source archive, which an 'ebuild' for that app uses to build and install the application as necessary.) > Some examples .... > - pkg-gentoo-LedgerSMB An "overlay" is not a package it's an ebuild and having the word "overlay' in the repo name for them is very commonly related to Gentoo. (Or so it seems to be judging from a search on GitHub...) -- Robert J. Clay rjclay(a)gmail.com -- Robert J. Clay rjclay(a)gmail.com

1 0

Cleaner tree for UI/ ?
by Erik Huelsmann 26 Aug '17

26 Aug '17

Hi, Since we moved to being a single-page app on the main app, I'm looking to improve clarity in the UI/ tree. What's the problem? I see a number of things being unclear: * Out of all the files stored in UI/, only these are actual "top level" documents: - UI/login.html - UI/logout.html - UI/main.html - setup/*.html * All files, also the bits that are "html fragments" these days, include a full html start- and end-tags as well as a header definition (which are being thrown away upon receipt!) * There are non-HTML documents (ODS, XLS, CSV, ...) in that directory (which suggests that it's not solely about UI, but also about data-export/report generation) Apart from the mix of HTML documents, the directory contains assets that *are* being served: javascript/images/css/... . Should we separate the HTML fragments in order to clarify the status as being a "non-toplevel document"? Another thing: I think we should separate out the templates which have not *just* an HTML representation, meaning that they are not (primarily) UI related elements. Regards, -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

3 5

Setting a policy for our dependencies
by Erik Huelsmann 26 Aug '17

26 Aug '17

Hi, Sending this mail to the mailing list, because I think that's where we need to set our policies. I'm starting this topic because I feel like it's come up several times now and I think it's time to set a policy, with documented reasons as to why we're working the way we do. First, let me re-iterate what I think are our shared goals for LedgerSMB (as related to our choice of dependencies). We want: LedgerSMB to be easy to install, meaning * On as many platforms as possible * Allowing people to use their distro's packages as much as possible * Running first and foremost on Perl versions in most distros * Allowing for a wide range of PostgreSQL versions * Integrating (through PSGI/reverse-proxy) with people's favorate webserver Translating the above into requirements for modules we depend on, I think we want to: * Use modules which have been around for a while (to increase chances of being packaged) * Allow the widest possible version range on any module, disallowing individual malfunctioning versions to further extend the range * Require as few a possible modules in the expanded dependency tree (prefer modules as direct dependencies which are already depended on implicitly) * Not depend directly on modules which have overlapping functionalities * Not require modules which only provide nice-to-have functionality * Group feature dependencies into their respective features as much as possible (so as to not require them for a more basic installation) With respect to modules used in development, I'd like to be less strict, but only marginally so: we want it to be "simple enough" to create a development setup for LedgerSMB. Comments? Regards, -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

1 0