Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot post html #172

Open
sgar7 opened this issue Dec 14, 2021 · 2 comments
Open

Cannot post html #172

sgar7 opened this issue Dec 14, 2021 · 2 comments
Labels
bug something isn't working

Comments

@sgar7
Copy link
Contributor

sgar7 commented Dec 14, 2021

Cannot post html (or tei) locally because it tries to webscrape first, and tries to connect to port:3001 which shouldn't happen.
Webscraping appears to be running even when you're just posting tei.

I ran this command: post -x html -t -f tei -o
and got this error message:
"Something went wrong while scraping the family letters website: URL(S): ["http://localhost:3001/research/maps/photographs", "http://localhost:3001/en/research/maps/photographs"]
ERROR: 500 Internal Server Error
To post content, please check the endpoint in config/public.yml, or temporarily disable the scrape_website setting in that file"

@sgar7 sgar7 added the bug something isn't working label Dec 14, 2021
@karindalziel
Copy link
Member

karindalziel commented Dec 14, 2021

It seems like no matter what filetype is being run (-f tei, above) it is trying to do the web scraping. But the web scraping won't work until after the index is run and local files created, so there's no way to get this set up locally other than downloading something from a server. More investigation needed, but also web scraping needs to be defined better in datura, probably.

@nichgray
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants