[Datasets] Add glossary

[bveeramani]

Signed-off-by: Balaji Veeramani balaji@anyscale.com

Why are these changes needed?

Related issue number

Closes #30408

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[maxpumperla]

@bveeramani didn't we want a "global" glossary for all terms in Ray? This is great, just trying to understand the direction

[bveeramani]

@bveeramani didn't we want a "global" glossary for all terms in Ray? This is great, just trying to understand the direction

I think a global glossary makes sense. Should we add this Datasets glossary for now, and later consolidate the terms in a global glossary? Not sure when we'll add terms for the other libraries.

[c21]

didn't we want a "global" glossary for all terms in Ray?

I thought we want to add glossary section under each library? cc @richardliaw to confirm, thanks.

[c21]

shall we also add an example for map_batches(batch_format=...)?

[bveeramani]

I don't think we should. Meaningful map_batches examples are relatively complicated, and I don't think we should include long examples in the glossary. Also, since we link to UDF Input Batch Formats, I think we should be okay.

[richardliaw]

Let's start here and then we can adjust it as more libraries start adding. @c21 @bveeramani

[bveeramani]

Failing tests are unrelated:

book-linkcheck

(ray-contribute/docs: line 10144) broken    https://github.com/ray-project/ray/blob/master/doc/source/tune/api_docs/integration.rst - 404 Client Error: Not Found for url: https://github.com/ray-project/ray/blob/master/doc/source/tune/api_docs/integration.rst
--
  | (ray-contribute/docs: line 10113) broken    https://github.com/ray-project/ray/blob/master/doc/source/tune/api_docs/overview.rst - 404 Client Error: Not Found for url: https://github.com/ray-project/ray/blob/master/doc/source/tune/api_docs/overview.rst
  | (ray-overview/index: line   18) broken    https://www.datanami.com/2018/02/01/rays-new-library-targets-high-speed-reinforcement-learning/ - 403 Client Error: Forbidden for url: https://www.datanami.com/2018/02/01/rays-new-library-targets-high-speed-reinforcement-learning/
  | (ray-overview/index: line    9) broken    https://www.datanami.com/2019/11/05/why-every-python-developer-will-love-ray/ - 403 Client Error: Forbidden for url: https://www.datanami.com/2019/11/05/why-every-python-developer-will-love-ray/
  | (rllib/rllib-algorithms: line  848) broken    https://lczero.org/ - 522 Server Error:  for url: https://lczero.org/
  | make: *** [Makefile:175: linkcheck] Error 1

book-documentation

File "../../python/ray/data/dataset.py", line ?, in default
--
  | Failed example:
  | ray.data.range_table(100).std("value", ddof=0)
  | Expected:
  | 28.86607004772212
  | Got:
  | 28.866070047722115
  | 2023-02-13 13:16:19,404	INFO bulk_executor.py:39 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[aggregate]
  | Shuffle Map: 100% 100/100 [00:00<00:00, 696.27it/s]
  | Shuffle Reduce: 100% 1/1 [00:00<00:00, 40.11it/s]

[clarkzinzow]

LGTM overall, mostly formatting nits!

[clarkzinzow]

LGTM, this is an awesome addition, and has been sorely missed for a long time! 👏

[jianoaix]

Great to see this glossary!