Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BulkExport Days not returning all records #650

Open
chadlarsh opened this issue Feb 10, 2023 · 5 comments
Open

BulkExport Days not returning all records #650

chadlarsh opened this issue Feb 10, 2023 · 5 comments
Labels
status: help wanted requesting help from the community type: bug bug in the library

Comments

@chadlarsh
Copy link

chadlarsh commented Feb 10, 2023 •
edited
Loading

Issue Summary

When attempting to list the available BulkExport days, the library does not return all records. According to the documentation, if no limit is specified, all records should be returned. It only returns the first 10 records. If you manually specify a page size of 10, the library will return all records.

Code Snippet

This will NOT return all records

client.bulkexports.v1.exports("Messages").days.list()

This will return all records

client.bulkexports.v1.exports("Messages").days.list(page_size=10)

Exception/Log

No exception occurs.

Technical details:

  • twilio-python version: 7.12.0
  • python version: 3.8.14
@sbansla
Copy link
Contributor

sbansla commented Feb 16, 2023

Hi @chadlarsh,
I will be checking the issue. Will get back to you soon. +1s on the issue summary will help it move up the backlog.

@sbansla
Copy link
Contributor

sbansla commented Feb 20, 2023

I tested:
client.bulkexports.v1.exports("Messages").days.list(). --> returns only 10 records.
This issue seems to be specific to this api only.

@sbansla sbansla added the type: bug bug in the library label Feb 20, 2023
@AsabuHere AsabuHere added status: waiting for feedback waiting for feedback from the submitter and removed status: waiting for feedback waiting for feedback from the submitter labels Mar 7, 2023
@AsabuHere
Copy link
Contributor

This issue has been added to our internal backlog to be prioritized. Pull requests and +1s on the issue summary will help it move up the backlog.

@AsabuHere AsabuHere added the status: help wanted requesting help from the community label Mar 7, 2023
@bernardgardner
Copy link

Additional note here: setting limit=1000 results in 30 days being returned - thanks @chadlarsh for the note that setting page_size=10 works around this issue - the bug also impacts client.bulkexports.v1.exports("Messages").days.stream() with the same symptoms (same issue with exports("Calls") too) -

@bernardgardner
Copy link

This actually doesn't seem to be a python issue - I'm seeing the same behavior talking to the API via curl. Sending the request with no params to get defaults yields a response with next_page_url: null - supplying the query param ?PageSize=10 (or any other value between 2 and 30 inclusive) works as expected.

(twilio-audit) bernard.gardner@bernard twilio-audit % curl -X GET "https://bulkexports.twilio.com/v1/Exports/Messages/Days" \            
-u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1594  100  1594    0     0   6831      0 --:--:-- --:--:-- --:--:--  7022
{
  "meta": {
    "page": 0,
    "page_size": 50,
    "first_page_url": "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=50&Page=0",
    "previous_page_url": null,
    "url": "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=50&Page=0",
    "next_page_url": null,
    "key": "days"
  },
  "days": [
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-01",
      "resource_type": "Messages",
      "size": 1857
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-02",
      "resource_type": "Messages",
      "size": 2172
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-03",
      "resource_type": "Messages",
      "size": 3099
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-04",
      "resource_type": "Messages",
      "size": 3582
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-05",
      "resource_type": "Messages",
      "size": 4584
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-06",
      "resource_type": "Messages",
      "size": 4625
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-07",
      "resource_type": "Messages",
      "size": 4269
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-08",
      "resource_type": "Messages",
      "size": 3504
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-09",
      "resource_type": "Messages",
      "size": 2609
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-10",
      "resource_type": "Messages",
      "size": 3258
    }
  ]
}

(twilio-audit) bernard.gardner@bernard twilio-audit % for i in {1..50}
do                
echo -n "$i "; curl -s -X GET "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=$i" -u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN | jq -Mr .meta.next_page_url
done
1 null
2 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=2&Page=1&PageToken=ENMjAyMy0wNC0wMg%3D%3D
3 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=3&Page=1&PageToken=ENMjAyMy0wNC0wMw%3D%3D
4 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=4&Page=1&PageToken=ENMjAyMy0wNC0wNA%3D%3D
5 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=5&Page=1&PageToken=ENMjAyMy0wNC0wNQ%3D%3D
6 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=6&Page=1&PageToken=ENMjAyMy0wNC0wNg%3D%3D
7 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=7&Page=1&PageToken=ENMjAyMy0wNC0wNw%3D%3D
8 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=8&Page=1&PageToken=ENMjAyMy0wNC0wOA%3D%3D
9 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=9&Page=1&PageToken=ENMjAyMy0wNC0wOQ%3D%3D
10 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=10&Page=1&PageToken=ENMjAyMy0wNC0xMA%3D%3D
11 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=11&Page=1&PageToken=ENMjAyMy0wNC0xMQ%3D%3D
12 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=12&Page=1&PageToken=ENMjAyMy0wNC0xMg%3D%3D
13 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=13&Page=1&PageToken=ENMjAyMy0wNC0xMw%3D%3D
14 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=14&Page=1&PageToken=ENMjAyMy0wNC0xNA%3D%3D
15 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=15&Page=1&PageToken=ENMjAyMy0wNC0xNQ%3D%3D
16 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=16&Page=1&PageToken=ENMjAyMy0wNC0xNg%3D%3D
17 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=17&Page=1&PageToken=ENMjAyMy0wNC0xNw%3D%3D
18 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=18&Page=1&PageToken=ENMjAyMy0wNC0xOA%3D%3D
19 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=19&Page=1&PageToken=ENMjAyMy0wNC0xOQ%3D%3D
20 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=20&Page=1&PageToken=ENMjAyMy0wNC0yMA%3D%3D
21 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=21&Page=1&PageToken=ENMjAyMy0wNC0yMQ%3D%3D
22 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=22&Page=1&PageToken=ENMjAyMy0wNC0yMg%3D%3D
23 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=23&Page=1&PageToken=ENMjAyMy0wNC0yMw%3D%3D
24 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=24&Page=1&PageToken=ENMjAyMy0wNC0yNA%3D%3D
25 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=25&Page=1&PageToken=ENMjAyMy0wNC0yNQ%3D%3D
26 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=26&Page=1&PageToken=ENMjAyMy0wNC0yNg%3D%3D
27 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=27&Page=1&PageToken=ENMjAyMy0wNC0yNw%3D%3D
28 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=28&Page=1&PageToken=ENMjAyMy0wNC0yOA%3D%3D
29 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=29&Page=1&PageToken=ENMjAyMy0wNC0yOQ%3D%3D
30 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=30&Page=1&PageToken=ENMjAyMy0wNC0zMA%3D%3D
31 null
32 null
33 null
34 null
35 null
36 null
37 null
38 null
39 null
40 null
41 null
42 null
43 null
44 null
45 null
46 null
47 null
48 null
49 null
50 null

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
status: help wanted requesting help from the community type: bug bug in the library
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@bernardgardner @chadlarsh @AsabuHere @sbansla