Feature:
- Don't retry non-existing profiles
- More efficient scrolling
- Max concurrency increased to 3 by default
Fix:
- Login modal blocking the scrolling
- More resilient URL inputs and normalization
Feature:
- Revamp to Typescript and Crawlee
Fix:
- Hanging timers on CPU overload
Fix:
- Timeline v2 object
Feature:
- Added '#sort_index' to the output
- Updated README
Fix:
- Thread replies
Fix:
- Search results
Features:
- Update SDK 2
Bug fixes:
- User shape object for some profiles
Features:
- Update to SDK 1.3.1
Changes:
- Change default timeout values
- Retiring of broken sessions
- Deals with pinned tweets
- Add debug log
Bug fixes:
- Fix thread extraction
Features:
- Update to SDK 1.2.1
Fixes:
- New GraphQL format
Features:
- Update to SDK 1.1.2
- Recursive "People" search
- Tweaks to wording in README and INPUT schema
Bug fixes:
- Filter cookies that lead to never loading page / 401 error
- Fetch data from GraphQl responses
Features:
- Update to SDK 1.0.2
Fixes:
- Clicking on non-replies buttons
Features:
- Scrape replies of replies
Fixes:
- Improve scraping stability
Features:
- Add topics
- Add hashtags URLs
- Optimize end of listings
- Labels for outputScraperFunction for various scraper phases
Fixes:
- Deduplication of tweets
- Force retiring forever failing proxies
- Add mentions, symbols, URLs and hashtags to output
- Add threads/status links support
- BREAKING CHANGE: Format of the dataset has changed
- Search multiple terms at once, search hashtags and terms
- Enriched user profile information (some information are only available when logged in)
- Added minimum and max tweet dates
- Updated SDK version
- Custom data
- Powerful extend output / scraper function
- Remove the need to provide credentials
- Update SDK version
- Allow to filter profile tweets for own tweets or include replies
- Scrape faster when there's no login information
- Accept twitter URLs, handles or
@usernamesfor better user experience - Throws immediately if invalid handles are passed