jupyter_data_dir usage outside of JupyterApp

[ivanov]

This came out of change that @krassowski brought up we needed to back out in nbformat, where jupyter_data_dir was attempted to be used by itself, and attempt ended up not matching the behavior of JupyterApp (which ensures a directory is created) and that broke some code downstream:

should it be the responsibility of jupyter_data_dir in jupyter_core to ensure that the directory exists? Can we pass an optional JupyterApp to jupyter_data_dir that uses it's data_dir if not set to default? Should we eliminate jupyter_data_dir function from jupyter_core altogether, to avoid this kind of dance?

For example, just a quick (not thorough) usage search I see that jupyter_data_dir is used in

https://github.com/jupyter/jupyter_client/blob/0a19278ed8d4090080a44760498a38ad9d4e0d99/jupyter_client/connect.py#L323

https://github.com/jupyter/jupyter_client/blob/0a19278ed8d4090080a44760498a38ad9d4e0d99/jupyter_client/kernelspec.py#L144

https://github.com/jupyterlab/jupyterlab/blob/946f18cc1178155a3b979adf058918ab63fb9a60/jupyterlab/federated_labextensions.py#L389

https://github.com/jupyter-server/jupyter_server_fileid/blob/v0.9.0/jupyter_server_fileid/manager.py#L23

all of those cases would end up using a possibly different directory in cases where the data_dir is specified by JupyterApp (or even if the directory was the same, would possibly error if that directory did not exist).