August 2017 - devel - lists.freelock.com

RFC: Formalize templates interface and split out onto CPAN
by Chris Travers 31 Aug '17

31 Aug '17

Hi; I have been thinking of what else we can/should split out sometime soon to CPAN and it has occurred to me that the interface of the templates could probably be split out (and maybe the mailer too). Both of these however contain LedgerSMB-specific assumptions and would need to be rewritten to some extent to address these. However with the templates in particular, removing these assumptions would help us get rid of a lot of the CGI assumptions we are currently making. In particular, I am thinking of the automatic output options which I think should be abstracted away or even removed. A major justification for this is that reporting endpoints for my LedgerSMB API gateway are much harder to implement across a Dancer interface than they should be, because we assume we are operating in a CGI environment. I would like to do this for the following reasons: 1. I would really like to be able to use the reporting generation in areas outside of a CGI web application, and 2. It would be nice to be able to open up additional formats to third party developers saying "Put it on CPAN and tell people they can install it like so." I would like to suggest that over the next major version or two I work on refactoring out or abstracting away the LedgerSMB-specific and CGI-specific assumptions to the templating interface, provide a dynamic format registration mechanism, and the like. And then work towards breaking these off once this is complete. Are there any thoughts on this? -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

3 9

Returning only content-type header from LedgerSMB::Template
by Erik Huelsmann 31 Aug '17

31 Aug '17

With the idea to simplify our LedgerSMB::Template handler, by moving all headers (but the content-type) from there to LedgerSMB::PSGI, I went to task: In the referenced issue, you write >>> >>> The template should be responsible *only* to render the content and >>> provide a file mime type back with it. This means that the output of >>> template->render() should ... >>> >>> Assuming that "template" there corresponds to my definition of >>> "LedgerSMB Template Engine" (concluded from the fragment of the second >>> sentence), I really wonder what value the template engine is providing: our >>> template processor has a render method too; the *only* thing it lacks is >>> the content type return value. The content type can be easily added based >>> on the extension of the processed template in a generic handler elsewhere >>> though. >>> >> >> Right. >> >>> >>> In the same paragraph, you continue >>> >>> The Template MUST NOT assume HTTP, PSGI, or any other particular >>> input/output format. >>> >>> Could you have a look and https://github.com/ledgersmb/L >>> edgerSMB/blob/master/lib/LedgerSMB/Template.pm#L540 ? I think that that >>> achieves what you want. >>> >> >> Yes, though more specifically, I think if we *do not use* PSGI or output >> options we achieve something much better. What I want to achieve is a >> clearer separation of responsibility and I think this has cascading >> benefits through the rest of the application as well as making the logic >> more general. >> > > Ok. You're proposing the return value being a pair: the expanded template > and the mime-type. The current implementation is a list of headers (usually > just the mime-type), the expanded template and a success/fail indicator (a > triplet). > > If we want to go with just the pair, how are we going to report the > success/fail indicator? (In other words, how do we report error > conditions?) I'll move the handling of the cache control headers currently > in "render_to_psgi()" to LedgerSMB::PSGI. That's a positive outcome of this > discussion for sure. > I have local work-in-progress. I've successfully moved the characterset addition to (utf-8) to all "text/" mime-types as well as the headers disabling caching when "back button disabling" is enabled, to LedgerSMB::PSGI. However, I'm trying to move the Content-Disposition headers to LedgerSMB::PSGI, but that's not quite so easy: the disposition header uses knowledge from the template (the type and name of the report). This information is not available on the LedgerSMB::PSGI level. So, how do I meaningfully name the downloaded file in this new world order? Regards, -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

2 1

Re: [ledgersmb-devel] Setting a policy for our dependencies
by Chris Travers 30 Aug '17

30 Aug '17

As a couple notes, I would add the following expansions to this. "Allow the widest possible version range on any module, disallowing individual malfunctioning versions to further extend the range" I think we are better off documenting bugs in misbehaving versions of modules and perhaps offering a module check utility rather than disallowing certain versions of modules. I say this for two reasons: 1. Managing this list is annoying and deciding when to block a specific version of a module is going to be an annoying and political decision, and 2. A block on a module version will not prevent someone who has an older version of the module from upgrading to a blocked version since it is an install-time check and another piece of software could require a newer version and thus break the install-time check. So I would suggest deciding required versions by first-supported version and up, and providing a tool or just documentation for checking for other problems in the mean time. Secondly I would like to suggest that a focus on moving things to CPAN and breaking the application into components for better testability. "Require as few a possible modules in the expanded dependency tree (prefer modules as direct dependencies which are already depended on implicitly)" Additionally I think that when we spin things off we should shoot for modules which are simple in interface, within a year or two will likely be pretty close to bug free and unlikely to require dramatic modifications, and therefore keep spin-offs throttled so they do not cause undue chaos for packagers. " Not depend directly on modules which have overlapping functionalities" Big caveat here is that some modules that are used *because* they are overlapping might be in a different position. Moo/Moose is a good example and both are widely used enough that there is no harm in using Moo instead of Moose in spun-off modules. However we still have some old paradigms in the code that need to be cleaned up and we should also focus on this. "Not require modules which only provide nice-to-have functionality" Also, while I am happy to see optional modules used to provide nice-to-have functionality it is worth noting that there are two things that need to be guarded against here. The first are extra dependencies. The second is extra code complexity that comes with adding such options. I would say that both need to be justified before we should allow them and where we do allow them we should think about how to set up proper interfaces so we don't have lots of conditional module handling everywhere. "Group feature dependencies into their respective features as much as possible (so as to not require them for a more basic installation)" A corollary here might be that long-run a lot of the application today should be relegated to optional features as we go forward. If someone doesn't need inventory tracking, why install it? If one doesn't need more than GL, customers, vendors, and basic reporting, why install more than that? Of course we aren't there yet but I would suggest that a forward-thinking view of dependencies we might want to think about it not only regarding the dependency problems we encounter with maintaining and working on the current code but also in the question of how modularised and loosely coupled we may want the code to be in the future. Best Wishes, Chris Travers -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

4 11

Sharing experiences packaging for Gentoo
by Chris Travers 29 Aug '17

29 Aug '17

Hi For various reasons I have started to package some other software for Gentoo and decided to put together an overlay for LedgerSMB. I wanted to share my experiences here and also looking at where the efforts I have seen on Debian, etc where we can probably make some improvements. First the good: Packaging the dependencies for LedgerSMB from CPAN was super-easy and straight-forward. It is as simple as using g-cpan to generate an initial ebuild, editing that to ensure that we aren't over-promising, and then writing a metadata.xml. I spent around an hour packaging all of our span dependencies. It was really, really easy. As a side note, one of the impressions I get from Jame (please correct me if I am wrong) is that one of the headaches here is legal: making sure we have all the right copyright information on all the right documentation, and this right now is a manual process. I will come back to that later. Packaging LedgerSMB is, on the other hand, very different. On one hand, these are not shared libraries so the build cannot go in vendor perl, and I am trying to support running multiple versions in parallel. Managing the direct dependency tree itself has been as much effort as packaging all of the dependencies. My approach is basically to: 1. Create an ebuild 2. Create a makefile which will overwrite the project makefile 3. Clobber the existing makefile with the new one in the ebuild 4. Point it at a directory (/usr/share/ledgersmb-1.4 for example) 5. Set up Plack (probably Starman at first. 6. Use an OpenRC init script (similar to Sys-V) to start Starman with each of the installed versions Right now I am on stage 2. Since there is a presumption against bundling, I will use any existing dojo ebuilds and this makes some things harder as well. However already packaging LedgerSMB has proved much more effort than packaging all the dependencies combined. One of the reasons I would like to see more stuff on CPAN from us is it would probably simplify this a lot. It's easier to have 2-3 optional dependencies each with 4 dependencies than to have 8-12 optional dependencies in 2-3 groups. Also, tracking which options have been enabled poses some complexity. From a Gentoo packaging perspective, the more we can put on CPAN the better. (Yes, g-cpan is not perfect, but it handles 90% of the cases nearly perfectly and once you get an ebuild, updating to a new version is super-easy). On to where I see some issues with the spinoff modules. 99% of the time I have been contacted by packagers (usually Jame) over packaging questions it is the nuts and bolts of things like copyright dates, changelog entries, etc. I would like to suggest that these can be subject to automatic testing. I would like to suggest the following global rules we can enforce by Travis or a git hook: 1. Copyright dates *MUST* include the current year for *all* modules changed, checked on every commit to master. This is something maybe we could enforce with a git hook. 2. Every tag MUST have a have an associated changelog entry. Again, something we could enforce via a git hook. There is one specific gentoo situation which is that source urls, being what they are, do not support backpan and cpan transparently together. Therefore I have to keep the overlay current regarding all currently supported versions and change urls to backpan were needed. That's a minor issue though. On the whole I expect working Gentoo packages in not too much longer and look forward to how we can help make packaging less work. -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

1 0

customizations
by R. Ransbottom 27 Aug '17

27 Aug '17

Hello. Sorry to be away. Various infections and physical traumas have left me doing only the basics for about seven weeks. That seems to be passing. Re: 1.5.9 What are recommendations for customizing and adding tables (and all that follows)? Tools? Naming conventions? etc. For parts, adding about 25 fields and 4 constraining tables similar to salutations. Lesser invasions to vendors and customer data. Suggestions and references will also be appreciated. Thanks, Rob

1 0

When to factor out code into a stand-alone CPAN release?
by Erik Huelsmann 27 Aug '17

27 Aug '17

Hi, In a recent issue, Chris proposed to factor out part of the code base into a stand-alone CPAN release (which would IMO mean into a more-or-less stand alone project). The same happened some years ago when he factored out DBObject into PGObject, so, we have precedence here. The other way around (refactored our code to use releases from CPAN) has happened as well. >From my perspective, the reasons to refactor our code to make use of (pre-existing) CPAN code are clear: * Using pre-existing code from CPAN outsources maintenance (to an external maintainer) * Opens up opportunities for wider use in our own code base, because usually replaces specific with generic code * Ensures better testing of code in a wide variety of settings (hopefully with bugfixes) * Improves code-sharing in the wider Perl community (hopefully free-ing up resources to take Perl and its modules a step further) To me, the other way around would roughly work the same way. Meaning, that we should be thinking about placing our code on CPAN, if: * Our code is sufficiently generic to solve problems for others * Others can depend on us to do the maintenance of the code * Our code comes with sufficient quality assurance measures to be helpful to others * Our code is sufficiently stable not to require large refactorings to the projects (which would mean that people can't really depend on the code - yet) Especially the last point - I think - is tricky: when largish refactorings are in order, the fact that part of the code being refactored is loaded onto Travis CI through CPAN is a major limitation on the speed that can be had during the refactoring. The recent PGObject 1.x->PGObject 2.x refactoring showed that. I'm wondering: Is there anything else to be considered? (Reasons to factor out, problems to be met after having separated, etc...) Regards, -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

2 2

ledgersmb project owned repository naming
by David G 26 Aug '17

26 Aug '17

Hi, Currently we have 30 repositories in the LedgerSMB github project. over half of them are related to PGObject modules and are reasonably named in the form of PGObject-foo-bar However, not all of our repositories are named so sensibly. In particular we now have 4 "packaging" repositories - pkg-ledgersmb - pkg-ledgersmb-1.4 - pkg-ledgersmb-1.5 - lsmb-overlay From the names, it's obscure at best which distro etc each of those repositories is for, more confusing is the fact that the first 3 seem to be duplicates of each other, excepting for the fact that they target different releases of our code. The first 3 are for debian packages, while the fourth is the new repo Chris has added for gentoo packaging. To help with "discoverability" and generally make it more obvious what a repo is for, I'd propose a naming scheme along the following lines. $purpose-$project-$detail For true "project" repositories such as LedgerSMB, and PGObject projects $purpose may not make sense, but for others it likely does. Some examples - LedgerSMB - PGObject - PGObject-Type-DateTime - pkg-debian-LedgerSMB - pkg-gentoo-LedgerSMB - migration_test_data-LedgerSMB - docker-LedgerSMB - docker-LedgerSMB-selenium-grid - OPOS-LedgerSMB-integration - LedgerSMB-PHP-bindings I think it's probably early enough in the lifecycle of most of the repositories that could do with renaming to rename them now. I say that as the ones that would change have little use outside of our project controlled resources. As for having the pkg-ledgersmb-1.x repositories as well as the unversioned one, Unless Jame has a reason to do otherwise, I'd suggest we simply use tags in the pkg-ledgersmb repo instead. Consider the situation if we ended up with packaging repos for every different distro, and also every release series we do in the future, things would quickly become unmanageable

2 2

Fwd: ledgersmb project owned repository naming
by Robert J. Clay 26 Aug '17

26 Aug '17

Sorry, forgot to send to the mailing list as well... ---------- Forwarded message ---------- From: Robert J. Clay <rjclay(a)gmail.com> Date: Sat, Aug 26, 2017 at 2:30 PM Subject: Re: [ledgersmb-devel] ledgersmb project owned repository naming To: David Godfrey <info(a)sbts.com.au> David, On Sat, Aug 26, 2017 at 6:43 AM, David Godfrey <info(a)sbts.com.au> wrote: > Currently we have 30 repositories in the LedgerSMB github project. > over half of them are related to PGObject modules and are reasonably named > in the form of > PGObject-foo-bar > > However, not all of our repositories are named so sensibly. > > In particular we now have 4 "packaging" repositories > .... > - lsmb-overlay > > ...., while the fourth is the new repo Chris has added for gentoo packaging. For an 'ebuild' for Gentoo, which is not quite the same thing... (A "package" for gentoo looks to be just an application source archive, which an 'ebuild' for that app uses to build and install the application as necessary.) > Some examples .... > - pkg-gentoo-LedgerSMB An "overlay" is not a package it's an ebuild and having the word "overlay' in the repo name for them is very commonly related to Gentoo. (Or so it seems to be judging from a search on GitHub...) -- Robert J. Clay rjclay(a)gmail.com -- Robert J. Clay rjclay(a)gmail.com

1 0

Cleaner tree for UI/ ?
by Erik Huelsmann 26 Aug '17

26 Aug '17

Hi, Since we moved to being a single-page app on the main app, I'm looking to improve clarity in the UI/ tree. What's the problem? I see a number of things being unclear: * Out of all the files stored in UI/, only these are actual "top level" documents: - UI/login.html - UI/logout.html - UI/main.html - setup/*.html * All files, also the bits that are "html fragments" these days, include a full html start- and end-tags as well as a header definition (which are being thrown away upon receipt!) * There are non-HTML documents (ODS, XLS, CSV, ...) in that directory (which suggests that it's not solely about UI, but also about data-export/report generation) Apart from the mix of HTML documents, the directory contains assets that *are* being served: javascript/images/css/... . Should we separate the HTML fragments in order to clarify the status as being a "non-toplevel document"? Another thing: I think we should separate out the templates which have not *just* an HTML representation, meaning that they are not (primarily) UI related elements. Regards, -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

3 5

Setting a policy for our dependencies
by Erik Huelsmann 26 Aug '17

26 Aug '17

Hi, Sending this mail to the mailing list, because I think that's where we need to set our policies. I'm starting this topic because I feel like it's come up several times now and I think it's time to set a policy, with documented reasons as to why we're working the way we do. First, let me re-iterate what I think are our shared goals for LedgerSMB (as related to our choice of dependencies). We want: LedgerSMB to be easy to install, meaning * On as many platforms as possible * Allowing people to use their distro's packages as much as possible * Running first and foremost on Perl versions in most distros * Allowing for a wide range of PostgreSQL versions * Integrating (through PSGI/reverse-proxy) with people's favorate webserver Translating the above into requirements for modules we depend on, I think we want to: * Use modules which have been around for a while (to increase chances of being packaged) * Allow the widest possible version range on any module, disallowing individual malfunctioning versions to further extend the range * Require as few a possible modules in the expanded dependency tree (prefer modules as direct dependencies which are already depended on implicitly) * Not depend directly on modules which have overlapping functionalities * Not require modules which only provide nice-to-have functionality * Group feature dependencies into their respective features as much as possible (so as to not require them for a more basic installation) With respect to modules used in development, I'd like to be less strict, but only marginally so: we want it to be "simple enough" to create a development setup for LedgerSMB. Comments? Regards, -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

1 0

ledgersmb project owned repository naming
by David Godfrey 26 Aug '17

26 Aug '17

1 0

[devel] Turning off Coveralls commenting on PRs?
by Erik Huelsmann 26 Aug '17

26 Aug '17

Hi, Currently, all PRs are tested by Coveralls and Travis CI. Travis CI doesn't comment on our PRs, but Coveralls does. Initially, maybe, there wasn't a good way for Coveralls to indicate success/failure otherwise. However, since both Coveralls and Travis have the "tests in progress" section on the PR, every PR will show the latest Coveralls state right above the merge button. So, I propose we stop Coveralls from generating these comments. Anybody who does want the Coveralls notifications would need to create a coveralls account, add the LedgerSMB repo to his account and set up e-mail notification. With that, you'll receive mail notification of every Travis CI build with coverage information. If no objections by the end of Sunday, I'll modify the settings in Coveralls accordingly. -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

2 1

Looking at building Gentoo packages
by Chris Travers 24 Aug '17

24 Aug '17

Hi; For another project I have had to learn Gentoo packaging. I think having a LedgerSMB overlay for Portage would be very helpful in attracting development effort, and I don't expect this to be a difficult one to maintain. I could probably manage to automate everything from build testing to actual builds on our releases. However, this leads to a few questions: 1. For Gentoo should we support multiple side-by-side versions (I.e. slotting with 1.4 and 1.5 occupying different slots)? If we do, should we automate having these share a starman instance? 2. My plan is to add the CPAN dependencies unspotted and assume they may be shared. 3. From here I plan to talk with a Gentoo maintainer to see if we can get the overlap to have some sort of official status and made available out of the box by the tool layman. Any thoughts or objections? -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

1 0

Re: [ledgersmb-devel] Primary key for tables "eca_to_contact", "entity_to_contact", "entity_bank_account"
by David G 23 Aug '17

23 Aug '17

Your proposed change looks good to me. Sent from my Samsung Galaxy Note 4 on the Telstra Mobile network -------- Original message -------- From: Erik Huelsmann <ehuels(a)gmail.com> Date: 23/08/2017 05:22 (GMT+08:00) To: devel(a)lists.ledgersmb.org Subject: [ledgersmb-devel] Primary key for tables "eca_to_contact", "entity_to_contact", "entity_bank_account" Hi, While working on some of the 1.5.10-planned fixes, I'm running into the problem that the three tables in the subject have primary keys which contain non-numeric data. More specifically, in case of the bank account data, the pkey contains the bic and account number and in case of the contact data, the pkey contains a user-description of the contact item in the pkey (together with a class). Knowing about the privacy protection regulations in the EU, I think that having the bic+iban in the pkey of a table which is to describe that data isn't practical (both because it already exposes the primary data through the pkey and because it exposes heavily sensitive data to all tables which want to enable looking this data up. For the contact table, things are a bit different: I'm thinking that having an (editable) user description in the pkey simply isn't a great idea. Proposed way forward: All three tables get a serial "id" column, which is made the PKEY, where we can opt to make a UNIQUE index out of the columns which currently determine the PKEY. Comments? -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP.Robust and Flexible. No vendor lock-in.

1 0

Primary key for tables "eca_to_contact", "entity_to_contact", "entity_bank_account"
by Erik Huelsmann 22 Aug '17

22 Aug '17

Hi, While working on some of the 1.5.10-planned fixes, I'm running into the problem that the three tables in the subject have primary keys which contain non-numeric data. More specifically, in case of the bank account data, the pkey contains the bic and account number and in case of the contact data, the pkey contains a user-description of the contact item in the pkey (together with a class). Knowing about the privacy protection regulations in the EU, I think that having the bic+iban in the pkey of a table which is to describe that data isn't practical (both because it already exposes the primary data through the pkey and because it exposes heavily sensitive data to all tables which want to enable looking this data up. For the contact table, things are a bit different: I'm thinking that having an (editable) user description in the pkey simply isn't a great idea. Proposed way forward: All three tables get a serial "id" column, which is made the PKEY, where we can opt to make a UNIQUE index out of the columns which currently determine the PKEY. Comments? -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

1 0

Test version ledgersmb-1.5 v1.5.9-2~1
by Robert J. Clay 14 Aug '17

14 Aug '17

On Sun, Aug 13, 2017 at 7:05 PM, Robert J. Clay <rjclay(a)gmail.com> wrote: > Note that this version does not include ledgersmb-1.5-apache for > the Apache proxy configuration as there was still an issue with that A test package of ledgersmb-1.5 that does include the problem ledgersmb-1.5-apache deb was also uploaded to our apt repository last night. That's in 'test' instead of 'main' and is version 1.5.9-2~1. That's also associated with the temporary branch 'dev/ApacheProxy' at https://github.com/ledgersmb/pkg-ledgersmb-1.5, as well as the open issue https://github.com/ledgersmb/pkg-ledgersmb-1.5/issues/2. At our apt package repository in 'test', for 'stretch': ledgersmb-1.5_1.5.9-2~1_all.deb ledgersmb-1.5-apache_1.5.9-2~1_all.deb ledgersmb-1.5-nginx_1.5.9-2~1_all.deb I emphasize that this is only a test version (and may not end up being the only one, depending on how the troubleshooting goes) for an issue with the apache proxy configuration associated with the legdersmb-1.5-apache package. Once that issue has been resolved, it will be released in a new non-test version. -- Robert J. Clay rjclay(a)gmail.com jame(a)rocasa.us 4096/198CAB6F43B7EA9A

1 0

Release of ledgersmb-1.5 v1.5.9-1
by Robert J. Clay 13 Aug '17

13 Aug '17

The initial version of the debian pkg for legdgersmb-1.5, version 1.5.9-1, has been released and uploaded to our apt package repository for 'main' in stretch. It has also been uploaded to our PPA, for artful (the future v17.10). Note that this version does not include ledgersmb-1.5-apache for the Apache proxy configurtion as there was still an issue with that but does include ledgersmb-1.5-nginx for the Nginx proxy configuration. At our apt package repository: ledgersmb-1.5_1.5.9-1_all.deb ledgersmb-1.5-nginx_1.5.9-1_all.deb At our PPA: ledgersmb-1.5_1.5.9-1~ubuntu17.10.1_all.deb ledgersmb-1.5-nginx_1.5.9-1~ubuntu17.10.1_all.deb -- Robert J. Clay rjclay(a)gmail.com jame(a)rocasa.us 4096/198CAB6F43B7EA9A

1 0

[ledger-smb-devel] Storing test-data for migrations?
by Erik Huelsmann 11 Aug '17

11 Aug '17

Hi, As part of a change that I'm implementing for LedgerSMB 1.2/1.3 migrations (to 1.5/1.6), I'd like to create a number of LedgerSMB database dumps for the purpose of testing those migrations. I'm thinking that these schema files, if the scope of the tests is expanded to SL 2.8, 3.0 and 3.2, as well as tests including some *real* and *problematic* data, the test data set may become largish or even plain large (multiple MBs to multiple 10s of MBs). While the test data sets *are* core to testing the software during development (and as part of the Continuous Integration efforts run on Travis CI), I think that putting these in each and every repository that's being cloned around the world is probably a bad idea: many of those will never be used for testing (instead they'll be used for tracking customized versions). Some years ago, we imported the full sources of some Dojo version, which we - even though long unused - we still drag around on every clone action. To that extent, I'm thining we probably want to store these artifacts elsewhere and keep them out of the repository. Maybe starting with a directory on download.ledgersmb.org. Any ideas/comments? -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

2 3

Excluding dotted files & directories in the release archive?
by Erik Huelsmann 11 Aug '17

11 Aug '17

Hi, Today I was looking at GitHub's option to add PR and Issue templates. Now that we have cleaned up our top-level directory, I'd rather *not* pollute it again with a largish series of documents that one of the cloud providers "requires". To that extent, GitHub offers the option to put the required documents in a ".github/" directory in the project's root. In the distribution archive, these documents wouldn't serve a purpose: the issue and PR templates only add value in the github site. The CONTRIBUTING and CODE_OF_CONDUCT files would be nothing more than pointers to our ledgersmb.org site where we publish our actual policies to that extent. I'm thinking we can remove all of the following in our release archives: - .github/ (when introduced) - .gitignore - .gitattributes - .gitmodules - .proverc - .travis.yml - .tx/ The .git/ directory already is being removed during the release process. Or should we just simply leave them in the distribution tar, letting everybody decide for him/herself and packagers for their users? -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

2 1

Re: [devel] Migration and Sanity checks
by David G 04 Aug '17

04 Aug '17

Sending this from my phone and being lazy, sorry for the top post. ;-) Ok. - I agree that both types of checks are required. - I think they should be two separate entry points in setup.pl just like upgrade is now. - At the end of a migration (or upgrade for that matter) There should be a screen that explains that we recommend running the sanity checks and provide a button to do so on that screen. Regards David G -------- Original message -------- From: Erik Huelsmann <ehuels(a)gmail.com> Date: 4/08/2017 22:49 (GMT+08:00) To: "Yves Lavoie, GaYLi" <ylavoie(a)yveslavoie.com> Cc: devel(a)lists.ledgersmb.org Subject: Re: [devel] Migration and Sanity checks On Fri, Aug 4, 2017 at 2:04 PM, Yves Lavoie, GaYLi <ylavoie(a)yveslavoie.com> wrote: Hi all, Until now, our migration process consisted of: Using an existing LSMB 1.2 to 1.5 or Sql-Ledger 2.8 or 3.0 database Applying a specific migration script, with various fix possibilities using specific forms Obtaining a dataset that can be pushed into the latest LedgerSMB schema or failing completely. During the last week, Erik and I have been working on adding different edit, update, insert, cancel and override possibilities for the migration forms, in order to ease migration and try to prevent returning the user to his old application to fix data.Yes. And looking back, we seem to be adding some checks which could be classified as "sanity checks" rather than "migration checks". The difference being that "sanity checks" are not required to pass for technical reasons (i.e. the data is acceptable for the current 1.5 or 1.6 schema); however, the data doesn't make sense: e.g. reconciliation entries on a cost account. My definition of "migration checks" is that these are checks which are required to pass in order for the target schema to be able to accept the incoming data. Yves's idea was to add these sanity checks during migration. My idea was that adding these sanity checks to the migration is the wrong place to add them: users who migrated before the checks were added might also have functionally problematic data, but can't find out, because they can't re-run the migration, if we put them *only* in the migration. Also, for people who don't really care about all this, they're required to completely clean up their data beyond what's necessary to migrate to LedgerSMB (making the migration experience itself more painful than necessary). Now, some of those fixing forms are specific to each migration but some are more general and should probably fit better in a new sanitization step, to be inserted between steps 2 and 3 above. As discussed with Yves on the chat channel, this is the part where we don't exactly agree yet. We completely agree that it makes sense to have sanity checks. But the questions I have are: 1. Do we require people to go through the sanitizing experience of having to comply with both the migration checks as well as the sanity checks? 2. Even if we do, how do we make the sanity checks available to other users (such as those who migrated in the past)? 3. If we decide to break out the sanity checks into a function of their own, where do we put that function? (setup.pl? something elsewhere?) 4. If we put them in a function of their own, we obviously can't put them in Upgrade_Tests.pm, but their completely parallels the upgrade tests. In terms of code re-use, do we extract out a common abstract superclass which also holds the driver(utility) functions, making the setup.pl equivalents into simple "forwarders"? This would benefit all previous versions prevent entering bad data in the database for which we do or not have means to fixMy idea here is that it only has meaning to have sanity checks when we have feedback to provide to the user, that is, when we can offer a resolution strategy (preferably different from "contact the LedgerSMB mailing lists"). In my eyes "bad" data isn't as bad as it looks. In the cases we've run into so far, "bad data" simply meant "functionally meaningless". provide a diagnostic of the database to the owner. Such report could also be generated on demand through the admin application. allow for more complex checks than was is strictly required by migration.And - more importantly - allow for much more complex "fix-up" operations than originally intended. This is a change from the actual process for which your input would be welcomed.Yup. Also see my questions and comments above. -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP.Robust and Flexible. No vendor lock-in.

1 0

[devel] Migration and Sanity checks
by Yves Lavoie, GaYLi 04 Aug '17

04 Aug '17

Hi all, Until now, our migration process consisted of: 1. Using an existing LSMB 1.2 to 1.5 or Sql-Ledger 2.8 or 3.0 database 2. Applying a specific migration script, with various fix possibilities using specific forms 3. Obtaining a dataset that can be pushed into the latest LedgerSMB schema or failing completely. During the last week, Erik and I have been working on adding different edit, update, insert, cancel and override possibilities for the migration forms, in order to ease migration and try to prevent returning the user to his old application to fix data. Now, some of those fixing forms are specific to each migration but some are more general and should probably fit better in a new sanitization step, to be inserted between steps 2 and 3 above. This would * benefit all previous versions * prevent entering bad data in the database for which we do or not have means to fix * provide a diagnostic of the database to the owner. Such report could also be generated on demand through the admin application. * allow for more complex checks than was is strictly required by migration. This is a change from the actual process for which your input would be welcomed. Yves

2 1