2024-06-17 18:37:09 [scrapy.utils.log] INFO: Scrapy 2.11.0 started (bot: gazette) 2024-06-17 18:37:09 [scrapy.utils.log] INFO: Versions: lxml 4.9.3.0, libxml2 2.10.3, cssselect 1.2.0, parsel 1.8.1, w3lib 2.1.2, Twisted 22.10.0, Python 3.10.7 (main, May 29 2023, 13:51:48) [GCC 12.2.0], pyOpenSSL 23.2.0 (OpenSSL 3.1.3 19 Sep 2023), cryptography 41.0.4, Platform Linux-5.19.0-46-generic-x86_64-with-glibc2.36 2024-06-17 18:37:09 [pr_mambore] INFO: Collecting data from 2024-05-01 to 2024-05-31. 2024-06-17 18:37:09 [scrapy.addons] INFO: Enabled addons: [] 2024-06-17 18:37:09 [py.warnings] WARNING: /home/marcos/Documentos/querido-diario/venv/lib/python3.10/site-packages/scrapy/utils/request.py:254: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy. See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation. return cls(crawler) 2024-06-17 18:37:09 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.epollreactor.EPollReactor 2024-06-17 18:37:09 [scrapy.extensions.telnet] INFO: Telnet Password: 99ddb8e1f963db79 2024-06-17 18:37:09 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats', 'spidermon.contrib.scrapy.extensions.Spidermon', 'gazette.extensions.StatsPersist'] 2024-06-17 18:37:09 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'gazette', 'COMMANDS_MODULE': 'gazette.commands', 'DOWNLOAD_TIMEOUT': 360, 'FILES_STORE_S3_ACL': 'public-read', 'LOG_FILE': 'log_pr_mambore_intervalo_maio_24.txt', 'NEWSPIDER_MODULE': 'gazette.spiders', 'SPIDER_MODULES': ['gazette.spiders'], 'TEMPLATES_DIR': 'templates', 'USER_AGENT': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:108.0) ' 'Gecko/20100101 Firefox/108.0'} 2024-06-17 18:37:09 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy_zyte_smartproxy.ZyteSmartProxyMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2024-06-17 18:37:09 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2024-06-17 18:37:09 [scrapy.middleware] INFO: Enabled item pipelines: ['gazette.pipelines.GazetteDateFilteringPipeline', 'gazette.pipelines.DefaultValuesPipeline', 'gazette.pipelines.QueridoDiarioFilesPipeline', 'spidermon.contrib.scrapy.pipelines.ItemValidationPipeline', 'gazette.pipelines.SQLDatabasePipeline'] 2024-06-17 18:37:09 [scrapy.core.engine] INFO: Spider opened 2024-06-17 18:37:09 [gazette.database.models] INFO: Populating 'querido_diario_spider' table - Please wait! 2024-06-17 18:37:09 [gazette.database.models] INFO: Populating 'querido_diario_spider' table - Done! 2024-06-17 18:37:09 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2024-06-17 18:37:09 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023 2024-06-17 18:37:13 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:14 [tzlocal] DEBUG: /etc/timezone found, contents: America/Sao_Paulo 2024-06-17 18:37:14 [tzlocal] DEBUG: /etc/localtime found 2024-06-17 18:37:14 [tzlocal] DEBUG: 2 found: {'/etc/timezone': 'America/Sao_Paulo', '/etc/localtime is a symlink to': 'America/Sao_Paulo'} 2024-06-17 18:37:14 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%221%22%7D%7D) 2024-06-17 18:37:18 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:18 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-24', 'edition_number': '923', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221084%22%2C%22hash%22%3A%22B23C984A492B6D7E88967338E3808EF3DF3FD969%22%7D&cidade=padrao'], 'files': [{'checksum': '96acc230b0a6189dbc066cf2244d9bfb', 'path': '4114005/2024-05-24/1e3c152971a0d149bf4cb7bf35598800abfd1610.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221084%22%2C%22hash%22%3A%22B23C984A492B6D7E88967338E3808EF3DF3FD969%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.636392Z', 'territory_id': '4114005'} 2024-06-17 18:37:18 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:18 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%221%22%7D%7D> {'date': '2024-05-29', 'edition_number': '926', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221087%22%2C%22hash%22%3A%22D01E7B36C17DFE9E1B2EBACEE6D7EF1445F59EBA%22%7D&cidade=padrao'], 'files': [{'checksum': '2c16367224bcebf3cbf17530f2b46797', 'path': '4114005/2024-05-29/88b2532666d1a77c219767a832855f76b3dd7cd8.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221087%22%2C%22hash%22%3A%22D01E7B36C17DFE9E1B2EBACEE6D7EF1445F59EBA%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.131328Z', 'territory_id': '4114005'} 2024-06-17 18:37:20 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:20 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-20', 'edition_number': '919', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221080%22%2C%22hash%22%3A%2246BB869AFE998C4105D480472339DDB485860681%22%7D&cidade=padrao'], 'files': [{'checksum': 'e769ee6f6aa7ff6567f16389dcc2d938', 'path': '4114005/2024-05-20/bc7769318a213e4829d2a96540c29f01357cf95b.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221080%22%2C%22hash%22%3A%2246BB869AFE998C4105D480472339DDB485860681%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.647348Z', 'territory_id': '4114005'} 2024-06-17 18:37:21 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:21 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-23', 'edition_number': '922', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221083%22%2C%22hash%22%3A%225F3AF728733AB048E9B3767422FB6BE7F3FE5C59%22%7D&cidade=padrao'], 'files': [{'checksum': '01435c9bd542ee82581bd443e8169df3', 'path': '4114005/2024-05-23/eed54f3e629e9467a82b0c1536f9aa1626d115b4.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221083%22%2C%22hash%22%3A%225F3AF728733AB048E9B3767422FB6BE7F3FE5C59%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.640542Z', 'territory_id': '4114005'} 2024-06-17 18:37:23 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:23 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:23 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%221%22%7D%7D> {'date': '2024-05-28', 'edition_number': '925', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221086%22%2C%22hash%22%3A%22DED2CE45277CE89E1F6F7ECED0BC72ED102A8296%22%7D&cidade=padrao'], 'files': [{'checksum': 'baac277154812dbc439023cc1385d027', 'path': '4114005/2024-05-28/66b2cdb991af6a05a679795bb1181456848ed301.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221086%22%2C%22hash%22%3A%22DED2CE45277CE89E1F6F7ECED0BC72ED102A8296%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.133596Z', 'territory_id': '4114005'} 2024-06-17 18:37:23 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:23 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:23 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-27', 'edition_number': '924', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221085%22%2C%22hash%22%3A%221AE2F9FB59B9B97443B10DF81E3DC3DD25408808%22%7D&cidade=padrao'], 'files': [{'checksum': '354802e879ec0120963c34efe2916750', 'path': '4114005/2024-05-27/720da97a15f877869aff1aba4f64679369d9e0f8.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221085%22%2C%22hash%22%3A%221AE2F9FB59B9B97443B10DF81E3DC3DD25408808%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.628095Z', 'territory_id': '4114005'} 2024-06-17 18:37:26 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:26 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:26 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-17', 'edition_number': '918', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221079%22%2C%22hash%22%3A%2214116BA99FB30F40AB6E308083419677055C1B34%22%7D&cidade=padrao'], 'files': [{'checksum': '329eb63b8ed11eb2ce4ef138ec2d3666', 'path': '4114005/2024-05-17/ca33672913434f3b103141f967222f250a66cdaf.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221079%22%2C%22hash%22%3A%2214116BA99FB30F40AB6E308083419677055C1B34%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.649065Z', 'territory_id': '4114005'} 2024-06-17 18:37:26 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:26 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:26 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-15', 'edition_number': '916', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221077%22%2C%22hash%22%3A%2230B5E2E5F8DCF110BB08B402AE96E58FF079B1FA%22%7D&cidade=padrao'], 'files': [{'checksum': 'b5a792f1d8f74cee26c9b684db1c68d6', 'path': '4114005/2024-05-15/7cff58abb9aa1e08c240cbf0c39915013b73767f.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221077%22%2C%22hash%22%3A%2230B5E2E5F8DCF110BB08B402AE96E58FF079B1FA%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.651337Z', 'territory_id': '4114005'} 2024-06-17 18:37:28 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:28 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:29 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-21', 'edition_number': '920', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221081%22%2C%22hash%22%3A%2267C9CF82314AA40658DF775DBE1559B4F6ACE311%22%7D&cidade=padrao'], 'files': [{'checksum': 'eac755b61084ccbb2b6c2ff0b81ae0e6', 'path': '4114005/2024-05-21/9bb13fa5c4dabeedd14a9e9d47093252b91fe6fd.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221081%22%2C%22hash%22%3A%2267C9CF82314AA40658DF775DBE1559B4F6ACE311%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.645608Z', 'territory_id': '4114005'} 2024-06-17 18:37:29 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:29 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:30 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-16', 'edition_number': '917', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221078%22%2C%22hash%22%3A%22DCEDFF742E106E1BB8E5B805089DECEC9EF0D891%22%7D&cidade=padrao'], 'files': [{'checksum': '2a644e2f30fb17d9071eb561880498db', 'path': '4114005/2024-05-16/15bfd00719ff2c25977b52e7d3769dd7eccd96e0.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221078%22%2C%22hash%22%3A%22DCEDFF742E106E1BB8E5B805089DECEC9EF0D891%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.650261Z', 'territory_id': '4114005'} 2024-06-17 18:37:30 [scrapy.core.engine] DEBUG: Crawled (200) (referer: https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D) 2024-06-17 18:37:31 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:31 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:31 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-22', 'edition_number': '921', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221082%22%2C%22hash%22%3A%22767727B15B73F8BAAA77EDF6715B30395FEBFDAA%22%7D&cidade=padrao'], 'files': [{'checksum': '58a030968117e67048f0508813db8442', 'path': '4114005/2024-05-22/79f12b1e807b9b28f422187655fcba4c7648f11e.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221082%22%2C%22hash%22%3A%22767727B15B73F8BAAA77EDF6715B30395FEBFDAA%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.643324Z', 'territory_id': '4114005'} 2024-06-17 18:37:31 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:31 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:31 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-10', 'edition_number': '913', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221074%22%2C%22hash%22%3A%22E87828255E8DE859B237DF4C27FCFA837C2C84C3%22%7D&cidade=padrao'], 'files': [{'checksum': 'da155d4de0d8bc3ea7c0bdf7417a6b5b', 'path': '4114005/2024-05-10/2077f66a7c79670a6b9bfa7e2205f0cd00ee673a.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221074%22%2C%22hash%22%3A%22E87828255E8DE859B237DF4C27FCFA837C2C84C3%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.654596Z', 'territory_id': '4114005'} 2024-06-17 18:37:37 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:37 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:37 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-13', 'edition_number': '914', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221075%22%2C%22hash%22%3A%2228778EFD0373BBBA0832119EBF17F1E51AB6ED6C%22%7D&cidade=padrao'], 'files': [{'checksum': '8f69b78e6ce4a8bbf4f5069a0d5a04bd', 'path': '4114005/2024-05-13/a628c0899575bd0f65df13f03ab427f8b6e14e38.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221075%22%2C%22hash%22%3A%2228778EFD0373BBBA0832119EBF17F1E51AB6ED6C%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.653527Z', 'territory_id': '4114005'} 2024-06-17 18:37:38 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:38 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:38 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-08', 'edition_number': '911', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221072%22%2C%22hash%22%3A%227BFE168510C14B4D94F1012A1832B102F6847EDA%22%7D&cidade=padrao'], 'files': [{'checksum': 'd61c5f38da57098bac0b56ee6ca4890b', 'path': '4114005/2024-05-08/0263c537be823954ff5441de74d6f86f7e658bb9.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221072%22%2C%22hash%22%3A%227BFE168510C14B4D94F1012A1832B102F6847EDA%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.657009Z', 'territory_id': '4114005'} 2024-06-17 18:37:39 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:39 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:39 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-09', 'edition_number': '912', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221073%22%2C%22hash%22%3A%220FA029B147B2D39357CF2EB0B21D61AC780DD462%22%7D&cidade=padrao'], 'files': [{'checksum': '12dbcbd886262d5f5bd13742059d58cb', 'path': '4114005/2024-05-09/7d96cedc2122c688f886e2b1f0db9dfdf6ea0d0b.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221073%22%2C%22hash%22%3A%220FA029B147B2D39357CF2EB0B21D61AC780DD462%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.655781Z', 'territory_id': '4114005'} 2024-06-17 18:37:40 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:40 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:40 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-14', 'edition_number': '915', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221076%22%2C%22hash%22%3A%22E4090BAC55F8E726DBB9A81CEC43706D24A92377%22%7D&cidade=padrao'], 'files': [{'checksum': '3a25ff94a4fc4e85ed8cc2df76874644', 'path': '4114005/2024-05-14/f0bf2dc9bad0016460bcca209e207b5668293c14.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221076%22%2C%22hash%22%3A%22E4090BAC55F8E726DBB9A81CEC43706D24A92377%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.652430Z', 'territory_id': '4114005'} 2024-06-17 18:37:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:42 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:42 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:42 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:42 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%223%22%7D%7D> {'date': '2024-05-02', 'edition_number': '907', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221068%22%2C%22hash%22%3A%22599459644186482543FADEB1048E4EA2D33C5C1D%22%7D&cidade=padrao'], 'files': [{'checksum': '6427c55a24e3b85ea9f61ef8d36f9605', 'path': '4114005/2024-05-02/6020c49e16ca235ce6d4f2b3efd1f205e56706e2.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221068%22%2C%22hash%22%3A%22599459644186482543FADEB1048E4EA2D33C5C1D%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:30.318994Z', 'territory_id': '4114005'} 2024-06-17 18:37:42 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%223%22%7D%7D> {'date': '2024-05-06', 'edition_number': '909', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221070%22%2C%22hash%22%3A%22ED39921641450ED3E396A521CF8E8223CAD8F9F4%22%7D&cidade=padrao'], 'files': [{'checksum': '8a029f0b6e742e132726d3d460a7a22f', 'path': '4114005/2024-05-06/00e68ef9fcd5daa43759a62e5e6692c2f2d18fc7.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221070%22%2C%22hash%22%3A%22ED39921641450ED3E396A521CF8E8223CAD8F9F4%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:30.280907Z', 'territory_id': '4114005'} 2024-06-17 18:37:44 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:44 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:44 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None) 2024-06-17 18:37:44 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from referred in 2024-06-17 18:37:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%222%22%7D%7D> {'date': '2024-05-07', 'edition_number': '910', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221071%22%2C%22hash%22%3A%227DFAE185480BC7DA3D08188F09E5E70F21A8D606%22%7D&cidade=padrao'], 'files': [{'checksum': 'e44d4381a4a266c3aec34b8460e11b29', 'path': '4114005/2024-05-07/2bef47772e46e40f7612183f8203654b7b11c0d8.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221071%22%2C%22hash%22%3A%227DFAE185480BC7DA3D08188F09E5E70F21A8D606%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:14.658144Z', 'territory_id': '4114005'} 2024-06-17 18:37:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://mambore.atende.net/diariooficial/edicao/pagina/atende.php?rot=54015&aca=101&ajax=t&processo=loadPluginDiarioOficial¶metro=%7B%22codigoPlugin%22%3A1%2C%22filtroPlugin%22%3A%7B%22pagina%22%3A%223%22%7D%7D> {'date': '2024-05-03', 'edition_number': '908', 'file_urls': ['https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221069%22%2C%22hash%22%3A%22E092282B01E5FED22F9DC6BF4C3A4BBB5349E51A%22%7D&cidade=padrao'], 'files': [{'checksum': '32a45cf2333fe6e8685a2f3ee2e3c3e5', 'path': '4114005/2024-05-03/da111b7e4f4b57fd776883b60556d2f51becc3ad.pdf', 'status': 'downloaded', 'url': 'https://mambore.atende.net/atende.php?rot=54002&aca=737&processo=download¶metro=%7B%22codigo%22%3A%221069%22%2C%22hash%22%3A%22E092282B01E5FED22F9DC6BF4C3A4BBB5349E51A%22%7D&cidade=padrao'}], 'is_extra_edition': False, 'power': 'executive_legislative', 'scraped_at': '2024-06-17T21:37:30.317111Z', 'territory_id': '4114005'} 2024-06-17 18:37:44 [scrapy.core.engine] INFO: Closing spider (finished) 2024-06-17 18:37:44 [scrapy.extensions.feedexport] INFO: Stored csv feed (20 items) in: pr_mambore_intervalo_maio_24.csv 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] ------------------------------ MONITORS ------------------------------ 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] Comparison Between Executions/Days without gazettes... OK 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] Requests/Items Ratio/Ratio of requests over items scraped count... OK 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] Error Count Monitor/test_stat_monitor... SKIPPED (Unable to find 'log_count/ERROR' in job stats.) 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] Finish Reason Monitor/Should have the expected finished reason(s)... OK 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] Item Validation Monitor/test_stat_monitor... SKIPPED (Unable to find 'spidermon/validation/fields/errors' in job stats.) 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] ---------------------------------------------------------------------- 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] 5 monitors in 0.011s 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] OK (skipped=2) 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] -------------------------- FINISHED ACTIONS -------------------------- 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] ---------------------------------------------------------------------- 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] 0 actions in 0.000s 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] OK 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] --------------------------- PASSED ACTIONS --------------------------- 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] ---------------------------------------------------------------------- 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] 0 actions in 0.000s 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] OK 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] --------------------------- FAILED ACTIONS --------------------------- 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] ---------------------------------------------------------------------- 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] 1 action in 0.000s 2024-06-17 18:37:44 [pr_mambore] INFO: [Spidermon] OK (skipped=1) 2024-06-17 18:37:44 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 11659, 'downloader/request_count': 23, 'downloader/request_method_count/GET': 23, 'downloader/response_bytes': 165312533, 'downloader/response_count': 23, 'downloader/response_status_count/200': 23, 'elapsed_time_seconds': 34.797779, 'feedexport/success_count/FileFeedStorage': 1, 'file_count': 20, 'file_status_count/downloaded': 20, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2024, 6, 17, 21, 37, 44, 262660, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 70043, 'httpcompression/response_count': 3, 'item_scraped_count': 20, 'log_count/DEBUG': 67, 'log_count/INFO': 34, 'log_count/WARNING': 1, 'memusage/max': 123179008, 'memusage/startup': 123179008, 'request_depth_max': 2, 'response_received_count': 23, 'scheduler/dequeued': 3, 'scheduler/dequeued/memory': 3, 'scheduler/enqueued': 3, 'scheduler/enqueued/memory': 3, 'spidermon/validation/fields': 160, 'spidermon/validation/items': 20, 'spidermon/validation/validators': 1, 'spidermon/validation/validators/item/jsonschema': True, 'start_time': datetime.datetime(2024, 6, 17, 21, 37, 9, 464881, tzinfo=datetime.timezone.utc)} 2024-06-17 18:37:44 [scrapy.core.engine] INFO: Spider closed (finished)