Craweler not fetching all available components

[darsenault]

Thank you for developing this tool! Great idea and work for sure.

I noticed that the crawler is not pulling all available snippets/components.

➜  tailwindui-crawler node index.js
Processing /components/marketing/sections/heroes...
* Found 1 snippet
Writing ./output/components/marketing/sections/heroes/with_angled_image_on_right.html...

There are 3 snippets not just 1:

Thank you.

[kiliman]

Did you add your username and password to the .env file? When I see only 1 component, that typically means it failed authentication and is simply showing the free "preview" page which is very limited.

[darsenault]

I did create the .env. But, it appears I had a typo from a bad copy/paste. Recreating the file indeed fixed the problem. Your crawler works beautifully. My apologies for the bother. Thank you!

[kiliman]

I've updated the crawler to check for success/failure on authentication so you won't get false positives