2024-05-20 20:57:05 [scrapy.utils.log] INFO: Scrapy 2.11.0 started (bot: gazette)
2024-05-20 20:57:05 [scrapy.utils.log] INFO: Versions: lxml 4.9.3.0, libxml2 2.10.3, cssselect 1.2.0, parsel 1.8.1, w3lib 2.1.2, Twisted 22.10.0, Python 3.11.5 (main, Aug 25 2023, 13:19:53) [GCC 9.4.0], pyOpenSSL 23.2.0 (OpenSSL 3.1.3 19 Sep 2023), cryptography 41.0.4, Platform Linux-5.15.146.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
2024-05-20 20:57:05 [ba_gongogi] INFO: Collecting data from 2012-05-01 to 2013-05-01.
2024-05-20 20:57:05 [scrapy.addons] INFO: Enabled addons:
[]
2024-05-20 20:57:05 [py.warnings] WARNING: /home/claromes/development/okbr/querido-diario/.venv/lib/python3.11/site-packages/scrapy/utils/request.py:254: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.

It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.

See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
  return cls(crawler)

2024-05-20 20:57:05 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.epollreactor.EPollReactor
2024-05-20 20:57:05 [scrapy.extensions.telnet] INFO: Telnet Password: 66498863129b7ca7
2024-05-20 20:57:06 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.feedexport.FeedExporter',
 'scrapy.extensions.logstats.LogStats',
 'spidermon.contrib.scrapy.extensions.Spidermon',
 'gazette.extensions.StatsPersist']
2024-05-20 20:57:06 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'gazette',
 'COMMANDS_MODULE': 'gazette.commands',
 'DOWNLOAD_TIMEOUT': 360,
 'FILES_STORE_S3_ACL': 'public-read',
 'LOG_FILE': 'log_ba_gongogi_2012-2013.txt',
 'NEWSPIDER_MODULE': 'gazette.spiders',
 'SPIDER_MODULES': ['gazette.spiders'],
 'TEMPLATES_DIR': 'templates',
 'USER_AGENT': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:108.0) '
               'Gecko/20100101 Firefox/108.0'}
2024-05-20 20:57:06 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy_zyte_smartproxy.ZyteSmartProxyMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2024-05-20 20:57:06 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2024-05-20 20:57:06 [scrapy.middleware] INFO: Enabled item pipelines:
['gazette.pipelines.GazetteDateFilteringPipeline',
 'gazette.pipelines.DefaultValuesPipeline',
 'gazette.pipelines.QueridoDiarioFilesPipeline',
 'spidermon.contrib.scrapy.pipelines.ItemValidationPipeline',
 'gazette.pipelines.SQLDatabasePipeline']
2024-05-20 20:57:06 [scrapy.core.engine] INFO: Spider opened
2024-05-20 20:57:06 [gazette.database.models] INFO: Populating 'querido_diario_spider' table - Please wait!
2024-05-20 20:57:06 [gazette.database.models] INFO: Populating 'querido_diario_spider' table - Done!
2024-05-20 20:57:06 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2024-05-20 20:57:06 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2024-05-20 20:57:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.gongogi.ba.gov.br/Site/DiarioOficial> (referer: None)
2024-05-20 20:57:07 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.gongogi.ba.gov.br/Site/DiarioOficial> (referer: https://www.gongogi.ba.gov.br/Site/DiarioOficial)
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=284&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=284&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=283&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=283&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=282&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=282&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.gongogi.ba.gov.br/Site/DiarioOficial> (referer: https://www.gongogi.ba.gov.br/Site/DiarioOficial)
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=288&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=288&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=287&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=287&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=281&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=281&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=280&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=280&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-11-29. File Checksum: 6cb044a2dd5c3d1fd1b4e8c938b7a4b0. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-11-29',
 'edition_number': 284,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=284&c=277&m=0'],
 'files': [{'checksum': '6cb044a2dd5c3d1fd1b4e8c938b7a4b0',
            'path': '2911501/2012-11-29/76dd612801850468c53e2b1cefc5dec59c60fd57',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=284&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.979815Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-11-27. File Checksum: c046f839e43d9dadd00510909fe9b3ce. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-11-27',
 'edition_number': 283,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=283&c=277&m=0'],
 'files': [{'checksum': 'c046f839e43d9dadd00510909fe9b3ce',
            'path': '2911501/2012-11-27/44284ca7cef19944a1cf0f2c452f9113413a3f7d',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=283&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.982949Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-11-19. File Checksum: ae31bacbc19e2e801a35bd0cbd243dd5. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-11-19',
 'edition_number': 282,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=282&c=277&m=0'],
 'files': [{'checksum': 'ae31bacbc19e2e801a35bd0cbd243dd5',
            'path': '2911501/2012-11-19/69ac91d33fe48e108e095b719ca0c21cd244c5df',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=282&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.985581Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=289&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=289&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=285&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=285&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=279&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=279&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-12-10. File Checksum: 2bb10571e322c13e8f9dd2a2d6f12984. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-12-10',
 'edition_number': 288,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=288&c=277&m=0'],
 'files': [{'checksum': '2bb10571e322c13e8f9dd2a2d6f12984',
            'path': '2911501/2012-12-10/5e15381c826a0ffae91ae884bd4bdbfc6e3c9de5',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=288&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.967552Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=278&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=278&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-12-05. File Checksum: 5cb53f11436eb93f72b50f9e60d08e99. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-12-05',
 'edition_number': 287,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=287&c=277&m=0'],
 'files': [{'checksum': '5cb53f11436eb93f72b50f9e60d08e99',
            'path': '2911501/2012-12-05/95ce45a74b2fd0c6bf614d65e8ca69f100d6c8d9',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=287&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.970503Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=276&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=276&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-11-09. File Checksum: c26d34c8a422f4d035d4f20b62fe540c. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-11-09',
 'edition_number': 281,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=281&c=277&m=0'],
 'files': [{'checksum': 'c26d34c8a422f4d035d4f20b62fe540c',
            'path': '2911501/2012-11-09/b1b944f2c722a85bea66cb0f2c4a7bf8733df9e5',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=281&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.988393Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-11-07. File Checksum: f9dc42909ad766c2df850dfe72b4bc9e. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-11-07',
 'edition_number': 280,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=280&c=277&m=0'],
 'files': [{'checksum': 'f9dc42909ad766c2df850dfe72b4bc9e',
            'path': '2911501/2012-11-07/8ea16d9db0dc153fe894457fcbba44da9840f87c',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=280&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.989549Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=274&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=274&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-12-20. File Checksum: fd7c4f321873f9ebd4d68d7e8cefc67b. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-12-20',
 'edition_number': 289,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=289&c=277&m=0'],
 'files': [{'checksum': 'fd7c4f321873f9ebd4d68d7e8cefc67b',
            'path': '2911501/2012-12-20/0a63cc0d5aa1382e4dae1d0c91c6694f0ef7ded9',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=289&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.964339Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-11-30. File Checksum: 405f4a4729c0f80f879fca14144497bd. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-11-30',
 'edition_number': 285,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=285&c=277&m=0'],
 'files': [{'checksum': '405f4a4729c0f80f879fca14144497bd',
            'path': '2911501/2012-11-30/b994293c217e29d58a674d32db61f88c6f7a6f55',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=285&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.976476Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-10-17. File Checksum: 3bec3d1203337d38ec933ec47f88876d. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-10-17',
 'edition_number': 279,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=279&c=277&m=0'],
 'files': [{'checksum': '3bec3d1203337d38ec933ec47f88876d',
            'path': '2911501/2012-10-17/2ec17e03cfa6b71862f041077d77cdf8a3514d17',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=279&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.992756Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=275&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=275&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=271&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=271&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-10-15. File Checksum: 7722b901cf1f696920c78039f5c0c3ab. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-10-15',
 'edition_number': 278,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=278&c=277&m=0'],
 'files': [{'checksum': '7722b901cf1f696920c78039f5c0c3ab',
            'path': '2911501/2012-10-15/d9332296dfe470efbae0d7fb097a94548adde312',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=278&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.994622Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=273&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=273&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-09-05. File Checksum: 6c470e20316d7cfe4d16bade0f9ecf18. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-09-05',
 'edition_number': 276,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=276&c=277&m=0'],
 'files': [{'checksum': '6c470e20316d7cfe4d16bade0f9ecf18',
            'path': '2911501/2012-09-05/79de5bf13efefa4af2636ff2a8a8f7051248d11a',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=276&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.996895Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=269&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=269&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=272&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=272&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-08-21. File Checksum: ce1928863ea76457850efbfec39542e3. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-08-21',
 'edition_number': 274,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=274&c=277&m=0'],
 'files': [{'checksum': 'ce1928863ea76457850efbfec39542e3',
            'path': '2911501/2012-08-21/1bcc7e7d8acd50e819f65e2f5cdecfa6bc0e548f',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=274&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.001220Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=268&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=268&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-08-22. File Checksum: 2bfe70234c368145ec89b8b51d954c9c. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-08-22',
 'edition_number': 275,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=275&c=277&m=0'],
 'files': [{'checksum': '2bfe70234c368145ec89b8b51d954c9c',
            'path': '2911501/2012-08-22/e2a251cc075d552cb535c0b5d590428d12a7b6a1',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=275&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.000329Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-08-13. File Checksum: 9be275cc5999a67f04e184f2fa088d36. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-08-13',
 'edition_number': 271,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=271&c=277&m=0'],
 'files': [{'checksum': '9be275cc5999a67f04e184f2fa088d36',
            'path': '2911501/2012-08-13/2b8fb55bc7a80551caddacca6dcf2d754ecbb518',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=271&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.003326Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=266&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=266&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=264&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=264&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-08-15. File Checksum: 804d56c497523e402cda781fca9ab70b. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-08-15',
 'edition_number': 273,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=273&c=277&m=0'],
 'files': [{'checksum': '804d56c497523e402cda781fca9ab70b',
            'path': '2911501/2012-08-15/bf519f27fbf2066b7716b015dc212bde60d42c36',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=273&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.002075Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=265&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=265&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-08-09. File Checksum: 1b89d483f85ecdda682eefc260573f47. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-08-09',
 'edition_number': 269,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=269&c=277&m=0'],
 'files': [{'checksum': '1b89d483f85ecdda682eefc260573f47',
            'path': '2911501/2012-08-09/90a6388b03c085ff1f325d1ad947fb08361615c6',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=269&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.004744Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=263&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=263&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-08-15. File Checksum: 61f83755015ca4197057e2bf1d8c271b. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-08-15',
 'edition_number': 272,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=272&c=277&m=0'],
 'files': [{'checksum': '61f83755015ca4197057e2bf1d8c271b',
            'path': '2911501/2012-08-15/96205be9c6cbad34d6f17f347aba9a8adad769dc',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=272&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.002713Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-07-30. File Checksum: d7b8a2bedb4b2f2e63e162dc468724aa. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-07-30',
 'edition_number': 268,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=268&c=277&m=0'],
 'files': [{'checksum': 'd7b8a2bedb4b2f2e63e162dc468724aa',
            'path': '2911501/2012-07-30/1452de60970de82304d55e4b9beeba299f67ba02',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=268&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.005384Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=261&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=261&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-07-30. File Checksum: b9919c54bd7f589491c70c207a4ec4d8. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-07-30',
 'edition_number': 266,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=266&c=277&m=0'],
 'files': [{'checksum': 'b9919c54bd7f589491c70c207a4ec4d8',
            'path': '2911501/2012-07-30/b922d77e74782fc9649d74533ccde4af542ad33c',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=266&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.006656Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-07-23. File Checksum: acb45db3d12a8c1702534a9878b1692b. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-07-23',
 'edition_number': 264,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=264&c=277&m=0'],
 'files': [{'checksum': 'acb45db3d12a8c1702534a9878b1692b',
            'path': '2911501/2012-07-23/eed8f0f99c8f752fbffa6e8f20a71d2854500b5b',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=264&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.007918Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-07-30. File Checksum: 3651f58820257f289951314de73ce7ad. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-07-30',
 'edition_number': 265,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=265&c=277&m=0'],
 'files': [{'checksum': '3651f58820257f289951314de73ce7ad',
            'path': '2911501/2012-07-30/601736fac3acf42b5c598335c32968b258821a2a',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=265&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.007284Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-07-10. File Checksum: 607736cf75168aee2b2674682f85309b. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-07-10',
 'edition_number': 263,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=263&c=277&m=0'],
 'files': [{'checksum': '607736cf75168aee2b2674682f85309b',
            'path': '2911501/2012-07-10/c372f4d3f81b6ffb6a7d21d4df1250d49469babd',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=263&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.008604Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=260&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=260&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-06-29. File Checksum: 6ddcd3c03a7b1702434f8ea3db065f9d. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-06-29',
 'edition_number': 261,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=261&c=277&m=0'],
 'files': [{'checksum': '6ddcd3c03a7b1702434f8ea3db065f9d',
            'path': '2911501/2012-06-29/f047ffab17edcb573b8b187527e3bc149f53c44c',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=261&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.010075Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=270&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=270&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=258&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=258&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=262&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=262&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-06-14. File Checksum: c0d6d26f80d3179e1c1be4e371289454. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-06-14',
 'edition_number': 260,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=260&c=277&m=0'],
 'files': [{'checksum': 'c0d6d26f80d3179e1c1be4e371289454',
            'path': '2911501/2012-06-14/b1d60344b654483478477019067c0b53159aa847',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=260&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.011514Z',
 'territory_id': '2911501'}
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=257&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=257&c=277&m=0> referred in <None>
2024-05-20 20:57:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=256&c=277&m=0> (referer: None)
2024-05-20 20:57:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=256&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-08-13. File Checksum: 8730c2809dab33e8175c9d9b6dce06f9. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-08-13',
 'edition_number': 270,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=270&c=277&m=0'],
 'files': [{'checksum': '8730c2809dab33e8175c9d9b6dce06f9',
            'path': '2911501/2012-08-13/8206efc5a963060a05b2843230ae840b8aa8e1a2',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=270&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.004060Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=293&c=277&m=0> (referer: None)
2024-05-20 20:57:09 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=293&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-05-25. File Checksum: 03d73a4605ce6b28c344e93c239a3498. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-05-25',
 'edition_number': 258,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=258&c=277&m=0'],
 'files': [{'checksum': '03d73a4605ce6b28c344e93c239a3498',
            'path': '2911501/2012-05-25/5ee67abbbf1219cf76fa5b1e8865a20b0f5d4340',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=258&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.012963Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-07-06. File Checksum: 8dea1963aa3fe6562e04425e19ad6606. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-07-06',
 'edition_number': 262,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=262&c=277&m=0'],
 'files': [{'checksum': '8dea1963aa3fe6562e04425e19ad6606',
            'path': '2911501/2012-07-06/1db9936be25ae7601ec76242629dd1ed5549c285',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=262&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.009313Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=259&c=277&m=0> (referer: None)
2024-05-20 20:57:09 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=259&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-05-24. File Checksum: 7348eb760c5794649acece89e37d8e3a. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-05-24',
 'edition_number': 257,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=257&c=277&m=0'],
 'files': [{'checksum': '7348eb760c5794649acece89e37d8e3a',
            'path': '2911501/2012-05-24/a69e8505fb617863b4b7814e01dc520fd138b082',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=257&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.013729Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-05-03. File Checksum: 24f4a5e3b7a3306feb4df5150dea83c2. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-05-03',
 'edition_number': 256,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=256&c=277&m=0'],
 'files': [{'checksum': '24f4a5e3b7a3306feb4df5150dea83c2',
            'path': '2911501/2012-05-03/10ae1e65bc255186cf977c5a2130a5de33937fac',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=256&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.014490Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2013-03-28. File Checksum: 6d085bf3d3eaadc2e76ae707f0b60335. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2013-03-28',
 'edition_number': 293,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=293&c=277&m=0'],
 'files': [{'checksum': '6d085bf3d3eaadc2e76ae707f0b60335',
            'path': '2911501/2013-03-28/9b9e1aeb5b89d85f39876fe8c77ec84347e0d80d',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=293&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.440914Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=291&c=277&m=0> (referer: None)
2024-05-20 20:57:09 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=291&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=292&c=277&m=0> (referer: None)
2024-05-20 20:57:09 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=292&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=290&c=277&m=0> (referer: None)
2024-05-20 20:57:09 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=290&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-05-30. File Checksum: 64d081796c02163b107174095ce43d6f. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-05-30',
 'edition_number': 259,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=259&c=277&m=0'],
 'files': [{'checksum': '64d081796c02163b107174095ce43d6f',
            'path': '2911501/2012-05-30/5c9503467730f3a50972dd591d064e667bff8e0c',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=259&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.012312Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2013-01-09. File Checksum: b9ed3b14afbd13844a4ab395d576c7b3. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2013-01-09',
 'edition_number': 291,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=291&c=277&m=0'],
 'files': [{'checksum': 'b9ed3b14afbd13844a4ab395d576c7b3',
            'path': '2911501/2013-01-09/6d17e782511b738ce5eb3dc792293c28b99f1a2b',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=291&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.443520Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2013-02-22. File Checksum: 998179b1e670ae8269518f7aab24cd20. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2013-02-22',
 'edition_number': 292,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=292&c=277&m=0'],
 'files': [{'checksum': '998179b1e670ae8269518f7aab24cd20',
            'path': '2911501/2013-02-22/735fef7b5c71bfb3a0ae06fd5ba9054dfed2099d',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=292&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.442330Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2013-01-09. File Checksum: 725e67429524a8c7e50cec808ca05355. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2013-01-09',
 'edition_number': 290,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=290&c=277&m=0'],
 'files': [{'checksum': '725e67429524a8c7e50cec808ca05355',
            'path': '2911501/2013-01-09/54714e8c9c7fe640cf5a7776300ae2b50803a744',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=290&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.445135Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=286&c=277&m=0> (referer: None)
2024-05-20 20:57:09 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=286&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=277&c=277&m=0> (referer: None)
2024-05-20 20:57:09 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=277&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-12-04. File Checksum: 1a2f2e1746b9aa7d09b4dffb6b9c27e7. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-12-04',
 'edition_number': 286,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=286&c=277&m=0'],
 'files': [{'checksum': '1a2f2e1746b9aa7d09b4dffb6b9c27e7',
            'path': '2911501/2012-12-04/c35e2fd7ee3eded5bc7e87d6f51bd2f711f06ccb',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=286&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.973356Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://sai.io.org.br/Handler.ashx?f=diario&query=267&c=277&m=0> (referer: None)
2024-05-20 20:57:09 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://sai.io.org.br/Handler.ashx?f=diario&query=267&c=277&m=0> referred in <None>
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-09-29. File Checksum: e4292f3a71f16f87d7156124ecd1b021. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-09-29',
 'edition_number': 277,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=277&c=277&m=0'],
 'files': [{'checksum': 'e4292f3a71f16f87d7156124ecd1b021',
            'path': '2911501/2012-09-29/d35a939daee2680d3c73f6da31d1852e0f177947',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=277&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:07.995870Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [ba_gongogi] WARNING: Something wrong has happened when adding the gazette in the database. Date: 2012-07-30. File Checksum: e01144e0f08c9378c19586b231ee62d8. Details: ('(sqlite3.IntegrityError) UNIQUE constraint failed: gazettes.territory_id, gazettes.date, gazettes.file_checksum',)
2024-05-20 20:57:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.gongogi.ba.gov.br/Site/DiarioOficial>
{'date': '2012-07-30',
 'edition_number': 267,
 'file_urls': ['https://sai.io.org.br/Handler.ashx?f=diario&query=267&c=277&m=0'],
 'files': [{'checksum': 'e01144e0f08c9378c19586b231ee62d8',
            'path': '2911501/2012-07-30/74d0199a8e92376d3094959d1e29ec7984468aa9',
            'status': 'downloaded',
            'url': 'https://sai.io.org.br/Handler.ashx?f=diario&query=267&c=277&m=0'}],
 'is_extra_edition': False,
 'power': 'executive_legislative',
 'scraped_at': '2024-05-20T23:57:08.006069Z',
 'territory_id': '2911501'}
2024-05-20 20:57:09 [scrapy.core.engine] INFO: Closing spider (finished)
2024-05-20 20:57:09 [scrapy.extensions.feedexport] INFO: Stored csv feed (38 items) in: ba_gongogi_2012-2013.csv
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] ------------------------------ MONITORS ------------------------------
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] Comparison Between Executions/Days without gazettes... OK
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] Requests/Items Ratio/Ratio of requests over items scraped count... OK
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] Error Count Monitor/test_stat_monitor... SKIPPED (Unable to find 'log_count/ERROR' in job stats.)
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] Finish Reason Monitor/Should have the expected finished reason(s)... OK
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] Item Validation Monitor/test_stat_monitor... SKIPPED (Unable to find 'spidermon/validation/fields/errors' in job stats.)
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] ----------------------------------------------------------------------
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] 5 monitors in 0.010s
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] OK (skipped=2)
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] -------------------------- FINISHED ACTIONS --------------------------
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] ----------------------------------------------------------------------
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] 0 actions in 0.000s
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] OK
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] --------------------------- PASSED ACTIONS ---------------------------
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] ----------------------------------------------------------------------
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] 0 actions in 0.000s
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] OK
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] --------------------------- FAILED ACTIONS ---------------------------
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] ----------------------------------------------------------------------
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] 1 action in 0.000s
2024-05-20 20:57:09 [ba_gongogi] INFO: [Spidermon] OK (skipped=1)
2024-05-20 20:57:09 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 12877,
 'downloader/request_count': 41,
 'downloader/request_method_count/GET': 39,
 'downloader/request_method_count/POST': 2,
 'downloader/response_bytes': 29047996,
 'downloader/response_count': 41,
 'downloader/response_status_count/200': 41,
 'elapsed_time_seconds': 3.157721,
 'feedexport/success_count/FileFeedStorage': 1,
 'file_count': 38,
 'file_status_count/downloaded': 38,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2024, 5, 20, 23, 57, 9, 702286, tzinfo=datetime.timezone.utc),
 'httpcompression/response_bytes': 31655292,
 'httpcompression/response_count': 38,
 'item_scraped_count': 38,
 'log_count/DEBUG': 118,
 'log_count/INFO': 34,
 'log_count/WARNING': 39,
 'memusage/max': 128720896,
 'memusage/startup': 128720896,
 'request_depth_max': 1,
 'response_received_count': 41,
 'scheduler/dequeued': 3,
 'scheduler/dequeued/memory': 3,
 'scheduler/enqueued': 3,
 'scheduler/enqueued/memory': 3,
 'spidermon/validation/fields': 304,
 'spidermon/validation/items': 38,
 'spidermon/validation/validators': 1,
 'spidermon/validation/validators/item/jsonschema': True,
 'start_time': datetime.datetime(2024, 5, 20, 23, 57, 6, 544565, tzinfo=datetime.timezone.utc)}
2024-05-20 20:57:09 [scrapy.core.engine] INFO: Spider closed (finished)