New data/collection API

[MoOx]

Currently our api is mostly a legacy from the initial version and is not really simple and pretty even if "it works". We have some limitations and some stuff are not optimal

• we have a single collection (that's not a big deal for me, since you can easily filter but I understand this can looks like a limitation) Add support for multiple collection #791
• the entire collection data is in all HTML pages (we could move into a file and load it before starting the client side rendering, but again it's still not optimal ): we need to have a way to split in HTML only what is used Improve collection loading for huge websites #609 Improve the way we consume collection(s) to avoid sending all metadata in all pages #713 Improve collection item private metadata #712
• it's not very easy to consume json files or external API (we could improve loader for json consumption but we will still be limited somehow) FAQ Consume more than one folder for the content #806 Generate page from json directly #358 Be able to consume data files #780 Add a way to consume Wordpress API #719 Add support and/or documentation for Contentful #886 Support .js and .json in phenomicLoader out of the box #880
• we publicly rely on react context, which is not good and intuitive (lot's of people say "it's not working" and they just forget to add contextTypes. We should hide this behind a provider component Consider dropping the usage of React context #304

I also want to add this

• we publicly use redux (some people are happy, but pretty sure most people don't care). We should hide what we use somehow Consider replacing redux in favor of react context #40
• we render content to html string which have some limitation, we should support more output, LIKE REACT COMPONENTS FOR EXAMPLE Use JS instead of JSON for pages #434
• routes should be resolved using not just markdown front-matter data Generate routes for data coming in from outside phenomic #732

Currently we could use more tricks but we discussed already about smarter ideas on how to have a better intuitive, minimalist API which will even unlock feature like pagination #657

⚠️ I didn't mention graphql since I don't think it will be really intuitive for most people. Gastby 1.0 branch is probably doing awesome stuff with that, but I have to admit I found that a bit overkill, especially for simple websites or documentation. It's probably very powerful but maybe a bit too much for most cases when you have data in collection(s) and just want to iterate on it (blog, photo gallery etc). You tell me.

---

Fresh API ideas have been exposed here #713 (comment)

I will start to work on something based on what @bloodyowl developed in this thread but feel free to comment and dissuss here about the approach YOU think is the best. There is no bad ideas, just ideas we can learn from ;)

[MoOx]

I am thinking about rewriting Phenomic using Next.js.

Pros:

• We will remove lots of code (dev server)
• We have an explicit way to differentiate static (server) and client rendering
• We will rely on a very active project with more users than us
• Ability to not explicitly rely on react, so you can use preact or inferno Add react-require to avoid importing React  vercel/next.js#295 + Support for pluggable renderers vercel/next.js#20
• Programmatic routes Programmatic API vercel/next.js#291
• html head support more intuitive than react-helmet api
• smart Link component that prefetch all resources when component is "around" you

Cons:

• I don't see how to handle optimistic loading (Add a loading indicator between <Link /> navigations vercel/next.js#238) for now, but maybe I didn't think hard enough yet
• No more react-router (anyway, v4 will make react-router support very limited as we just cannot statically render all routes (as recursive routes are possible))
• default files babel and webpack config might not work out of the box Custom babel config vercel/next.js#26 & Custom webpack config vercel/next.js#40

Next will offer an official API to handle static rendering vercel/next.js#70

---

Anyway, we might probably stop relying on a loader to consume markdown|* files with probably a simple interface like metalsmith that will still allow a plugin API.
Why? Because we need something generic to allow consuming files or promise (rest api, graphql api...)

[bloodyowl]

Going to be fixed with #925