Hi,
Talking on our chat channel, we were talking how to best implement our API.
Over the past months, the idea was to implement our new API as JSON:API.
This mail serves to lay out experiences and thoughts so far. Maybe it helps
to determine good next steps.
As a first step, it would probably be a good idea to list why we need an
API followed by what we need/want from it. The Why's:
* Integration with other applications, not all provided by the project
* Better support the needs from our web front-end: Vue, Dojo, react all
want to be hooked to REST APIs
* Development of custom front-ends
What would we like our API to be?
1. Standards compliant where we can; if a generic browser could be used to
"surf" the API, even better
2. Preferably well-defined semantics for good guidance on implementation
3. Resource definitions don't mirror internal table structures, but define
logical concepts
4. Easy to understand and use by non-project developers
5. Easy to expand by project developers while maintaining consistency
6. Development supported by Perl libraries on CPAN
7. Supports returning the same resource in many forms (e.g. invoice as
JSON, PDF or HTML)
From the investigations we did, it seems JSON:API supports (1), (2), (5)
and (6): it provides good guidance on the structure of resource URL naming
as well as the structure of the JSON object returned. There's a even PONAPI
on CPAN: Perl's implementation of a JSON:API server.
With respect to (3) [resource definitions], I'm finding the standard
extremely limiting: due to the structure imposed on returned documents,
references to other resources are always "top level" members. I mean that
the standard allows resources to be complex structures, with arrays and
objects nested in the definition. However, those nested structures can't
refer to other resources. As an example: if the lines of a GL journal is
included in the resource definition as an array, then the lines themselves
can't refer to other resources, such as accounts or batches. The standard
authors solve this by making the lines separate (nested) resources, which
are returned with the request for the journal. With this limitation, the
resources in the api quickly start to mirror the internal table structure:
GL lines need to refer accounts, invoice lines need to refer to parts and
inventory. Orders need to refer to multiple addresses (billing, shipping).
Journal lines need to refer to batches, payments, transactions, accounts,
etc. Most examples where JSON:API explains related and nested resources, it
takes a blog post with its comments. For that use-case, I understand the
structure chosen. However, in the use-case of a GL transaction, the
transaction *is* its lines.
On the topic of (7) [support for returning multiple forms of the resource],
I can be short: allowing any other mime-type on a JSON:API endpoint than
application/vnd.api+json is a violation of the spec.
Then there's (6) [supported by Perl libraries]. The good thing are: it's a
PSGI app that can be mounted in our URL space and building the JSON:API
server is work we don't have to do anymore. It seems pretty low on activity
though: its latest release was in Feb 2019, with a bit of prior activity
in Dec 2017, Feb 2017 and Sep 2016. The number of open issues is pretty
low: 13, but none newer than 2016; from the open issues, it's clear that it
doesn't support the full specification yet, because it doesn't support
paging of "referred to" resources. The issue that's been untouched the
longest (
https://github.com/mickeyn/PONAPI/issues/45) calls for thorough
investigation before we jump on this module, however: "Creating a
repository is non-trivial" (a repository in PONAPI is where the data is
sourced from); apparently, implementing a data source is still a lot of
work, even though the framework is there. Another downside that I found
from PONAPI is that it doesn't pass the web-request context on to the data
provider. Normally that isn't a problem, likely, but for our project, where
the web request context equals database access, this is a big problem.
In short:
* PONAPI is incomplete and development is slow (but has releases on CPAN
dating back to 2015, so maybe it's mostly "done")
* JSON:API has -by design- some serious limitations in the way we can
define our resources, resulting - quite likely - in mirroring internal
table structure in the resource definition
* our desire to return various representations of the same resource
conflicts with the JSON:API standard
Given the above, I'd welcome your thoughts, opinions and reseach
contributions.
To address some (or all) of the short-comings of PONAPI, I've started
developing our own JSON:API server component. However, there is a lot of
work to be done to move that anywhere. I hope this mail will stir
discussion and provide direction on whether or not to continue on our own
JSON:API component, seeking contact with PONAPI or to move in a different
direction entirely.
Regards,
Erik.