Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] [Connector] Customization of the crawler settings #737

Closed
2 tasks done
hmtbr opened this issue Oct 24, 2024 · 0 comments
Closed
2 tasks done

[Feature] [Connector] Customization of the crawler settings #737

hmtbr opened this issue Oct 24, 2024 · 0 comments
Assignees
@hmtbr
Labels
enhancement New feature or request

Comments

@hmtbr
Copy link
Collaborator

hmtbr commented Oct 24, 2024

Search before asking

  • I searched the issues and found no similar issues.

Component

Other

Feature

The data-prep-connector doesn't allow customization of the crawler settings like request concurrency, delay and timeouts. We need to update the library so that the user can specify their own settings.

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!
@hmtbr hmtbr added the enhancement New feature or request label Oct 24, 2024
@hmtbr hmtbr self-assigned this Oct 24, 2024
@hmtbr hmtbr mentioned this issue Oct 24, 2024
Merged
@hmtbr hmtbr closed this as completed Oct 31, 2024
@hmtbr hmtbr mentioned this issue Nov 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hmtbr hmtbr

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@hmtbr