fix: leading character exploit

[grantdfoster]

The changes in the forwarder.py file primarily focus on handling response IDs in the TweetValidator class by using regular expressions to remove leading zeros or similar characters. This ensures that padded zeros are properly handled in the response IDs, which is crucial for maintaining consistency and accuracy in tweet identification and validation.

Detailed Description of Changes:

1. Regex for Removing Leading Zeros:

   • A regular expression (re.sub(r"^[0०]+", "", resp["Tweet"]["ID"])) is used to strip leading zeros from tweet IDs. This is applied when processing responses to ensure that tweet IDs are unique and correctly formatted.

2. Modification in get_miners_volumes Method:

   • The method now includes logic to handle tweet IDs by removing leading zeros before processing them. This is done to ensure that each tweet is uniquely identified and validated without issues caused by padded zeros.

3. Spot Check for Valid Tweets:

   • After processing the tweet IDs, a spot check is performed to validate a random tweet from the responses. This involves checking the presence of the query in the tweet's text, name, username, and hashtags, as well as ensuring the tweet's timestamp is within the requested date range.

4. Validation and Scoring:

   • Tweets that pass the spot check are considered valid, and their IDs are stored in a set to ensure uniqueness. The number of valid tweets is then used to update the volume score for each miner.

5. Logging Enhancements:

   • Additional logging has been added to provide feedback on the success or failure of the spot check and the number of valid tweets produced by each miner.

Ensure that tweet IDs are consistently formatted and free of leading zeros, which can cause issues in identification and validation processes. By using regex to remove these characters, the system can maintain a consistent and accurate record of tweet IDs, which is essential for the validation and scoring mechanisms in place.

This change is particularly important in environments where tweet IDs might be stored or transmitted with leading zeros, potentially leading to discrepancies or errors in processing. The use of regex provides a robust solution to this problem, ensuring that all tweet IDs are handled uniformly.

[github-actions]

PR description is too short and seems to not fulfill PR template, please fill in

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.67%. Comparing base (71965be) to head (dea0a6b).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #332      +/-   ##
==========================================
+ Coverage   62.65%   62.67%   +0.02%     
==========================================
  Files          23       23              
  Lines        1320     1321       +1     
==========================================
+ Hits          827      828       +1     
  Misses        493      493              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.