Hi. First, yes it is good to focus on definitions and consistent vocabulary. Explanation to follow, along with and will modify RFC to match. On Wed, Aug 30, 2017 at 8:57 PM, Erik Huelsmann <ehuels(a)gmail.com> wrote: Fair enough. Let's go with these terms. Right. Yes, though more specifically, I think if we *do not use* PSGI or output options we achieve something much better. What I want to achieve is a clearer separation of responsibility and I think this has cascading benefits through the rest of the application as well as making the logic more general. Yes, the LedgerSMB::PSGI or whatever replaces it in the future. In other words the wrapper should be the one to decide what headers to send (and should have all PSGI-related functionality). But looking at it, the PSGI.pm doesn't decide what headers to send does it? It gets this from the workflow script which gets it from the template engine, is that correct? I would like to work on getting rid of the render_to_psgi and the output functions from the template engine, and return the file type with the contents. I think to make this work, we would need to provide a compatibility class for old code. Here's a case in point: Suppose I want a cron job which emails a spreadsheet of overdue invoices (ageing detail report). I think this should be a fairly easy thing to do even with the global state requirements, but on 1.4 it goes something like "fork our reporting engine" and the problem still is not fully fixed in master (worse, it is not *obvious* that the problem is not fully fixed in master because of assumptions in the templating engine). Agreed. One of the point of trying to formalise things that can be broken off is to make good libraries and a smaller footprint. But this is about two things. The first is general interface and having an interface which avoids assuming that every component is being accessed from the whole framework. I think these two are closely connected and global state is a part of the second. The case that again immediately occurred to me here was the fact that LedgerSMB::Report->render sends the result in CGI format, despite the fact that this is a module introduced in 1.4. And even in master, you have a choice of PSGI or CGI only (because for historical reasons we output to cgi by default). Yes, we can fix it in the reporting engine by adding a line of code but I guess I see this as a single responsibility problem because the templating engine takes on the responsibility of output which I think is a problem. (But if it doesn't take on the responsibility to output then I don't think it should take on PSGI responsibility either.) There is a related question here which is where should the HTTP status messages come from but I think that's an additional discussion. I think they should only come from the workflow scripts or from PSGI.pm but open to other ideas. -- -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

On Thu, Aug 31, 2017 at 8:39 PM, Erik Huelsmann <ehuels(a)gmail.com> wrote: That's one too many options for the body in my view. It means unneeded complexity given that different formats will probably return differently here and you always have to handle both. I would like to see, ideally, one response format only. A file handle would be good if we standardised on that. Quick: does the csv and OOO adaptors return bodies with the same interface? Or do I have to find out what kind of reference the body is and handle it? And why does the template library have special knowledge of HTTP status? Yep. And the UI portion will probably eventually go away. Exactly what I am trying to get to. Right but Dancer's template engine assumes you are going to use it with Dancer, to generate web pages with HTML and jquery. I guess I am questioning that assumption for an accounting package. Right. So suggesting we a shim for old code and a clean separation of concerns. I guess that's the question, isn't it? Here's what I would like to see (and I think this goes well with the efforts to move to a more service-oriented front-end): I would like to see the template infrastructure be a report and document generator which abstracts generation of documents in a variety of formats for the programmer. That's one responsibility. It is a narrow one. We can build something that then works for the future by ignoring what happens after we generate the document. Put another way, what I see the module doing is providing an abstraction around "I want a report" and "btw, I want it in pdf format" or "I want it as a csv." Or "I want an invoice" and "Please make it a PDF." Now to do this it has to do two things: 1. Maintain knowledge of available formats on the system (and possibly be able to do so on a per-template basis, something we are currently missing), and 2. It has to provide a stable interface for format providers to use to communicate with it and to the program as a whole. This means that what we have now for Template.pm in my proposal is basically a point of dependency injection, which maintains the responsibility of being aware of what dependencies have used it (PDF, HTML, etc). The Template::LaTeX, Template::HTML, etc. then take on the responsibility of plugging in to allow document generation in a singularly consistent API. That sounds small and narrow and it is. But think about what that enables as well: 1. Dynamic format detection (What formats do I know about? PDF, HTML, PS, OOO, and Gnumeric*) 2. Dynamic template detection (What formats can give me an invoice with my current template list?) * hypothetical new format we just installed Note we have to provide a shim for old code. But I think the stabilisation and generalisation here would be helpful. More tangibly, it would allow someone to write an HTML -> PDF template engine third party, and just have it plugged in if that was still desired by anyone, or for someone who wants to be able to generate OOO doc invoices to be able to do so without demanding that everyone else support the functionality. At any rate that's what I had in mind. Best Wishes, Chris Travers -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more

On Thu, Aug 31, 2017 at 11:26 PM, Erik Huelsmann &lt;ehuels(a)gmail.com&gt; wrote: That's a good point and especially applicable to reports. One nice thing about file handles in Perl though is that they are an abstraction which means you don't have to know whether it has been loaded into memory or not. In all Perl versions we support you can open a file handle to a scalar reference to a string. See the following: Chriss-MBP:.ssh christravers$ perl use strict; use warnings; use 5.010; my $string = "This is\na long\nstring across\nmultiple lines"; open my $fh, '<', \$string; print "line: $_" for <$fh>; say " and we are done"; Prints: line: This is line: a long line: string across line: multiple lines and we are done So I guess with that the question becomes: Are there any cases where file handles introduce problems in this regard? Ok so aside from the file handle/array of strings issue we need to come up with a good error handling model. Our error handling model is worse than just that problem in that often our errors are depending on a locale in global state in order to work and that is something that makes it impossible to, for example, build a list of GL transactions out of LedgerSMB from a third party module and have some approximation of half-way sane error handling at all. Maybe we should bring that up in a separate thread. But this kind of gets at what I think is important in general in software engineering, which is that we build as much as possible in small, generally useful pieces that aren't going to change much and then we have a stable environment on which the rest of the project can move faster once some of those smaller pieces are off the table. To be honest the error handling is an even higher priority for me. Ok. Initial thought is that maybe we need a LedgerSMB::Output namespace where we put mailers, send to ui, etc and maybe have a LedgerSMB::Template::UI class which we then use in many of these cases for simplicity purpose. If we accept that it is always the web glue layer then, the implication of this is that reporting is a problem in a non-web environment right? Given the way we abuse the template engine in the old code I think an old code-compatible version of things will need to be available. Maybe we change the name of the package but part of the reason our concerns are still not clear or separate in the template engine is because the old code uses templates to, say, print to a printer which means our template engine knows about our configured printers and how to send files there. I am not sure they are entirely different, in that there is currently some overlap. One of the things that we do that SQL-Ledger did also because it works well from a user workflow perspective is allow someone to go through a report and do things to lines. A good example here would be the unapproved batches report where one can drill down into a batch (showing vouchers in a batch) but also select batches and post or delete them. So here is a document which exists in both leagues. It is a report (and is useful as a report) and it also provides a user interface. Now if you request the report in PDF format you obviously lose these (and so that is only in the reporting league). Long-run we could dojoize it, and build a UI around the report document in a different way, but currently there is not.a way to say that these represent documents with non-overlapping use cases. That's a good point. A whole lot of it can be, actually. And actually I think there is a whole lot of room for improvement on every level of this area. There is a coordination layer, however, that cannot be. In reporting this manifests as the list of formats you can select. The problem for the old code is in the printing logic. I think we need a good replacement for that problem too as we move forward. However, I also don't want to disturb all the old code. My thought is to have a LedgerSMB::Template::Old class that replaces the LedgerSMB::Template class in these cases. The template engine was a *major* step forward for 1.3 and the importance cannot be understated. But because of the way the old code handles printing, separating certain concerns there hasn't really been possible. Well, in this case I think it should be old-code-only and only to avoid breaking things which are known-brittle. The refactoring will be done when the old code is gone. Another option is we build something else and depreciate LedgerSMB::Template. That might actually be better. Maybe we call it LedgerSMB::Document? Or maybe we build it on CPAN first, and then decide if we want to use it? -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more