Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ingest Migration - OpenData Datasets #1247

Open
14 of 19 tasks
fvankrieken opened this issue Nov 15, 2024 · 15 comments · May be fixed by #1289
Open
14 of 19 tasks

Ingest Migration - OpenData Datasets #1247

fvankrieken opened this issue Nov 15, 2024 · 15 comments · May be fixed by #1289
Assignees
@sf-dcp

Comments

@fvankrieken
Copy link
Contributor

fvankrieken commented Nov 15, 2024 •
edited by sf-dcp
Loading

All 19 datasets in the opendata weekly pull should be migrated. This includes

  • dcp_edesignation_csv
  • doitt_buildingfootprints
  • doitt_buildingfootprints_historical
  • dot_projects_intersections
  • dot_projects_streets
  • dpr_capital_projects (script source, but is part of this recurring job)
  • dpr_forever_wild
  • dpr_greenthumb
  • dpr_park_access_zone
  • dpr_parksproperties
  • dpr_schoolyard_to_playgrounds
  • dsny_frequencies
  • dsny_garages
  • fdny_firehouses
  • lpc_historic_districts
  • lpc_historic_district_areas
  • lpc_landmarks
  • lpc_scenic_landmarks
  • nypd_policeprecincts
@sf-dcp
Copy link
Contributor

sf-dcp commented Nov 27, 2024

nypd_policeprecincts

________________________________________________________________________________
Tables
    Left: nypd_policeprecincts_library
    Right: nypd_policeprecincts_ingest
________________________________________________________________________________
Row count
    Left: 77
    Right: 77
________________________________________________________________________________
Column comparison
    Both
        data_library_version
        ogc_fid
        precinct
        shape_area
        shape_leng
        wkb_geometry
    Left only: None
    Right only: None
    Type differences
        Precinct
            Left: character varying
            Right: text
        Shape area
            Left: character varying
            Right: text
        Shape leng
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Compared columns
        precinct
        shape_area
        shape_leng
        wkb_geometry
    Left only
        Empty DataFrame
        Columns: [shape_area, shape_leng, wkb_geometry, precinct]
        Index: []
    Right only
        Empty DataFrame
        Columns: [shape_area, shape_leng, wkb_geometry, precinct]
        Index: []

@sf-dcp sf-dcp linked a pull request Nov 27, 2024 that will close this issue
Draft
@sf-dcp sf-dcp moved this from New to 🏗 In progress in Data Engineering Nov 27, 2024
@fvankrieken
Copy link
Contributor Author

So dpr_capitalprojects, two notes

  • there is an in progress ingest template in dcpy/lifecycle/ingest/dev_templates that has the json parsing (hopefully?) done
  • this really is unrelated to socrata, so while doing it with this group, let's remove it from the gha. I added it here a21cee2 for some reason, but this doesn't seem appropriate. I think I was just mistaken and added the wrong dataset to the job

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 2, 2024 •
edited
Loading

lpc_scenic_landmarks

________________________________________________________________________________
Tables
    Left: lpc_scenic_landmarks_library
    Right: lpc_scenic_landmarks_ingest
________________________________________________________________________________
Row count
    Left: 13
    Right: 13
________________________________________________________________________________
Column comparison
    Both
        borough
        caldate
        current_
        data_library_version
        desdate
        last_actio
        lp_number
        ogc_fid
        public_hea
        scen_lm_na
        shape_area
        shape_leng
        status
        wkb_geometry
    Left only
        the_geom
    Right only: None
    Type differences
        Public hea
            Left: character varying
            Right: text
        Current 
            Left: character varying
            Right: text
        Scen lm na
            Left: character varying
            Right: text
        Lp number
            Left: character varying
            Right: text
        Borough
            Left: character varying
            Right: text
        Status
            Left: character varying
            Right: text
        Last actio
            Left: character varying
            Right: text
        Shape area
            Left: character varying
            Right: text
        Shape leng
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Key columns
        lp_number
    Left only: None
    Right only: None
    Columns with diffs: None
    Differences by column: None

Notes:

  • Left only column present: library used to ingest this geospatial dataset as a csv, resulting in 2 duplicate geom columns, the_geom & wkb_geometry. In new ingest template, we are extracting it as a geojson and it has one geom column. The duplicate geom column the_geom from library isn't used downstream. Therefore, we can ignore the diff.

  • 2 date columns from library, caldate & desdate are extracted as date strings. In ingest, these columns are of date type which makes more sense so keeping them that way. These columns aren't used downstream.

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 2, 2024

lpc_landmarks

________________________________________________________________________________
Tables
    Left: lpc_landmarks_library
    Right: lpc_landmarks_ingest
________________________________________________________________________________
Row count
    Left: 38933
    Right: 38933
________________________________________________________________________________
Column comparison
    Both
        bbl
        bin_number
        block
        boroughid
        boundaries
        caldate
        count_bldg
        data_library_version
        desdate
        desig_addr
        hist_distr
        last_actio
        lm_name
        lm_type
        lot
        lp_number
        most_curre
        non_bldg
        ogc_fid
        other_hear
        pluto_addr
        public_hea
        secnd_bldg
        status
        vacant_lot
        wkb_geometry
    Left only
        the_geom
    Right only: None
    Type differences
        Lot
            Left: character varying
            Right: text
        Most curre
            Left: character varying
            Right: text
        Lp number
            Left: character varying
            Right: text
        Other hear
            Left: character varying
            Right: text
        Count bldg
            Left: character varying
            Right: text
        Lm type
            Left: character varying
            Right: text
        Secnd bldg
            Left: character varying
            Right: text
        Block
            Left: character varying
            Right: text
        Public hea
            Left: character varying
            Right: text
        Hist distr
            Left: character varying
            Right: text
        Bbl
            Left: character varying
            Right: text
        Vacant lot
            Left: character varying
            Right: text
        Boroughid
            Left: character varying
            Right: text
        Non bldg
            Left: character varying
            Right: text
        Boundaries
            Left: character varying
            Right: text
        Pluto addr
            Left: character varying
            Right: text
        Last actio
            Left: character varying
            Right: text
        Lm name
            Left: character varying
            Right: text
        Bin number
            Left: character varying
            Right: text
        Desig addr
            Left: character varying
            Right: text
        Status
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Key columns
        wkb_geometry
        lp_number
    Left only: None
    Right only: None
    Columns with diffs: None
    Differences by column: None

Notes (same as lpc_scenic_landmarks ):

  • Left only column present: library used to ingest this geospatial dataset as a csv, resulting in 2 duplicate geom columns, the_geom & wkb_geometry. In new ingest template, we are extracting it as a geojson and it has one geom column. The duplicate geom column the_geom from library isn't used downstream. Therefore, we can ignore the diff.

  • 2 date columns from library, caldate & desdate are extracted as date strings. In ingest, these columns are of date type which makes more sense so keeping them that way. These columns aren't used downstream.

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 3, 2024

lpc_historic_district_areas

________________________________________________________________________________
Tables
    Left: lpc_historic_district_areas_library
    Right: lpc_historic_district_areas_ingest
________________________________________________________________________________
Row count
    Left: 157
    Right: 157
________________________________________________________________________________
Column comparison
    Both
        area_name
        borough
        boundary_n
        caldate
        current_
        data_library_version
        desdate
        extension
        last_actio
        lp_number
        ogc_fid
        other_hear
        public_hea
        shape_area
        shape_leng
        status_of_
        wkb_geometry
    Left only
        the_geom
    Right only: None
    Type differences
        Extension
            Left: character varying
            Right: text
        Status of 
            Left: character varying
            Right: text
        Other hear
            Left: character varying
            Right: text
        Borough
            Left: character varying
            Right: text
        Shape area
            Left: character varying
            Right: text
        Lp number
            Left: character varying
            Right: text
        Shape leng
            Left: character varying
            Right: text
        Area name
            Left: character varying
            Right: text
        Public hea
            Left: character varying
            Right: text
        Current 
            Left: character varying
            Right: text
        Last actio
            Left: character varying
            Right: text
        Boundary n
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Compared columns
        area_name
        borough
        boundary_n
        caldate
        current_
        desdate
        extension
        last_actio
        lp_number
        other_hear
        public_hea
        shape_area
        shape_leng
        status_of_
        wkb_geometry
    Left only
        Empty DataFrame
        Columns: [extension, status_of_, other_hear, borough, shape_area, wkb_geometry, caldate, lp_number, shape_leng, area_name, public_hea, desdate, current_, last_actio, boundary_n]
        Index: []
    Right only
        Empty DataFrame
        Columns: [extension, status_of_, other_hear, borough, shape_area, wkb_geometry, caldate, lp_number, shape_leng, area_name, public_hea, desdate, current_, last_actio, boundary_n]
        Index: []

Notes (same as lpc_scenic_landmarks ):

  • Left only column present: library used to ingest this geospatial dataset as a csv, resulting in 2 duplicate geom columns, the_geom & wkb_geometry. In new ingest template, we are extracting it as a geojson and it has one geom column. The duplicate geom column the_geom from library isn't used downstream. Therefore, we can ignore the diff.

  • 2 date columns from library, caldate & desdate are extracted as date strings. In ingest, these columns are of date type which makes more sense so keeping them that way. These columns aren't used downstream.

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 3, 2024 •
edited
Loading

lpc_historic_districts

________________________________________________________________________________
Tables
    Left: lpc_historic_districts_library
    Right: lpc_historic_districts_ingest
________________________________________________________________________________
Row count
    Left: 37672
    Right: 37672
________________________________________________________________________________
Column comparison
    Both
        alt_arch_1
        alt_arch_2
        alt_date_1
        alt_date_2
        altered
        arch_build
        bbl
        bbl_int
        bin
        block
        borough
        build_nme
        build_oth
        build_type
        circa
        data_library_version
        date_combo
        date_high
        date_low
        des_addres
        hist_dist
        lm_new
        lm_orig
        lot
        mat_four
        mat_other
        mat_prim
        mat_sec
        mat_third
        notes
        ogc_fid
        own_devel
        sc_flag
        shape_area
        shape_leng
        style_oth
        style_prim
        style_sec
        use_orig
        use_other
        wkb_geometry
    Left only
        the_geom
    Right only: None
    Type differences
        Hist dist
            Left: character varying
            Right: text
        Notes
            Left: character varying
            Right: text
        Mat third
            Left: character varying
            Right: text
        Style sec
            Left: character varying
            Right: text
        Mat other
            Left: character varying
            Right: text
        Arch build
            Left: character varying
            Right: text
        Des addres
            Left: character varying
            Right: text
        Alt arch 2
            Left: character varying
            Right: text
        Alt date 1
            Left: character varying
            Right: text
        Borough
            Left: character varying
            Right: text
        Build nme
            Left: character varying
            Right: text
        Bin
            Left: character varying
            Right: text
        Alt date 2
            Left: character varying
            Right: text
        Mat four
            Left: character varying
            Right: text
        Circa
            Left: character varying
            Right: text
        Date combo
            Left: character varying
            Right: text
        Build type
            Left: character varying
            Right: text
        Block
            Left: character varying
            Right: text
        Sc flag
            Left: character varying
            Right: text
        Bbl
            Left: character varying
            Right: text
        Alt arch 1
            Left: character varying
            Right: text
        Shape leng
            Left: character varying
            Right: text
        Style prim
            Left: character varying
            Right: text
        Bbl int
            Left: character varying
            Right: text
        Shape area
            Left: character varying
            Right: text
        Style oth
            Left: character varying
            Right: text
        Use orig
            Left: character varying
            Right: text
        Own devel
            Left: character varying
            Right: text
        Altered
            Left: character varying
            Right: text
        Lot
            Left: character varying
            Right: text
        Date low
            Left: character varying
            Right: text
        Use other
            Left: character varying
            Right: text
        Mat prim
            Left: character varying
            Right: text
        Date high
            Left: character varying
            Right: text
        Build oth
            Left: character varying
            Right: text
        Lm orig
            Left: character varying
            Right: text
        Lm new
            Left: character varying
            Right: text
        Mat sec
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Key columns
        bbl
        wkb_geometry
    Left only
        ('2025040006', '0106000020E6100000010000000103000000020000006F000000963C33F7747B52C03D338F4B536A44409BBF46D2747B52C0E4D311AE536A44401797260A747B52C00388F7C3556A444038C928F5737B52C04E93D4B5556A444058216CA9727B52C0C8C279D6546A44408931EA73717B52C0BEF91406546A44405E5B5981717B52C0CC4EBEE3536A444054378EAD717B52C0700EBF72536A4440181360DB717B52C08164A2FD526A44409D0AA5FF717B52C0A5BEECA0526A44405B4EAB1C727B52C0FA58BE56526A4440E8BEAD39727B52C000AE970C526A4440E686133F727B52C0E955CCFE516A4440C17ABF56727B52C0A15A4AC2516A44407E13E127737B52C07A79BBAB4F6A4440E9F462DF737B52C01719AED64D6A44409E8DFAF2737B52C09F6E5FE84D6A444011F15947747B52C0D6638B344E6A44408967A099747B52C0BA3AD37E4E6A44400E689AC4747B52C05E29A1A54E6A44405CEF0D10757B52C0DFEEC5E44D6A4440149E2FF5747B52C05BC683CC4D6A444076E0D8A1747B52C0F81444814D6A44409A22014B747B52C0F8E1DC324D6A4440BAAC7034747B52C0650C7E1E4D6A44402C43711F747B52C051CC2A544D6A444070CB4A16737B52C08A4AC6644C6A444045F67F2C737B52C083F0042C4C6A4440EBA71F08737B52C0C2E72C0B4C6A444058402749737B52C0E4CBF6644B6A44403B09CD36737B52C0FAEE4B504B6A444079268626737B52C01419F0374B6A44405AFFA318737B52C03C275F1C4B6A444034CA6D0D737B52C0C87C22FE4A6A444031E91805737B52C08900D1DD4A6A4440EA8DD0FF727B52C0A4300DBC4A6A4440D15CAFFD727B52C00F537F994A6A4440426DBFFE727B52C0A086D3764A6A4440EC8FFB02737B52C072D4B7544A6A44402E4F4F0A737B52C0AC51D8334A6A444035649414737B52C0A151D9144A6A4440E3149921737B52C0CF8953F8496A44400FD61931737B52C0B0FED5DE496A444026C1CC42737B52C07829E2C8496A4440FB1F5656737B52C00F27E6B6496A4440BE6D566B737B52C0BADE38A9496A444022B36281737B52C0A2BB21A0496A444086700D98737B52C06315CD9B496A44400458E5AE737B52C0DB0D509C496A4440EDD877C5737B52C0BA91A8A1496A444094F154DB737B52C0175ABDAB496A4440CDE90DF0737B52C09B0D5ABA496A4440722A3931747B52C04226C713496A4440D9309855747B52C0C52B9F34496A444084854469747B52C014475602496A44404D910851757B52C0BEEB97D3496A4440CDEB6A72757B52C0BCB0BAF1496A4440284E3467757B52C020EC610E4A6A4440F9ED55E5757B52C084FB42804A6A44406330EE58767B52C0A5ECA1E84A6A4440FC41B1BF767B52C0314D68454B6A4440BA9B620D777B52C06DDB8C8B4B6A44407A414C4B777B52C0D91973C34B6A44408FBC5378777B52C0906D1BEC4B6A444015EEE349777B52C0E410CE624C6A4440C4762C32787B52C0D7A685344D6A4440EA1D3BA6787B52C085064F9D4D6A4440CBEFCEAF787B52C0975DF4A54D6A4440D0FED42B797B52C0AADDF2684C6A44404492F796787B52C0AD618CE24B6A4440B369B55C797B52C06B4B1AE9496A4440F8A34A88797B52C06B9873104A6A4440CCB269BD7A7B52C0C1EF8B274B6A4440C09456E17A7B52C0E758B8CB4A6A4440D22DB1E27A7B52C08B8B1FC94A6A444010A475EF7A7B52C0E0D7AFB04A6A4440AD15FBFF7A7B52C0A5C300994A6A44404F038F127B7B52C0E2C326854A6A444019A0D0267B7B52C02A278A754A6A4440F5F2543C7B7B52C009F37D6A4A6A4440F5EEAB527B7B52C011083C644A6A4440462D5F697B7B52C0DF31E3624A6A444046BEF57F7B7B52C0D4F57C664A6A44400ACEFB957B7B52C022EAF36E4A6A44408BFFFAAA7B7B52C07A4D1F7C4A6A444002E385BE7B7B52C0D170B78D4A6A4440046935D07B7B52C03B615FA34A6A4440266EABDF7B7B52C042E9A4BC4A6A4440DB1799EC7B7B52C0A07004D94A6A44400277B8F67B7B52C06FFAE8F74A6A4440F870D5FD7B7B52C0E116AE184B6A44407EECC9017C7B52C0888DA93A4B6A4440015E80027C7B52C0B97F275D4B6A44406C52F6FF7B7B52C0C725727F4B6A4440F1E239FA7B7B52C030BDD3A04B6A4440C5043BF67B7B52C08B4E38AF4B6A444073E265F17B7B52C0AB32A0C04B6A4440DB7843CD7B7B52C01808FD1C4C6A4440D3A3503D7C7B52C091BF26824C6A4440A6E892777B7B52C0E1FC987B4E6A44400FA6A1207B7B52C073C7182D4E6A4440ECB0C759797B52C0A1FFB9B7526A444031FBB8B0797B52C0B53F3A06536A4440037D8031797B52C028FF6A4B546A444017D903DF787B52C0DC67421E556A44404C185A99777B52C0AB533DF8536A444077429C31777B52C06E7A929A536A44400555C718777B52C0B52C0ADA536A4440107E90B3767B52C0A8DBC1DC546A4440AF4F6255757B52C0EAB396A0536A4440963C33F7747B52C03D338F4B536A44401900000010E12CFD747B52C076F14A3C536A4440621C692D767B52C0114BA532506A444021BAFB39767B52C08B128112506A4440D79B5A46767B52C0B540AB1D506A444077B6307C767B52C043B8484E506A44401439FB93767B52C095657911506A4440C3E78B6A777B52C08C7831D3506A4440B3A84EC3777B52C0A5BE4EF04F6A4440EFF44789777B52C07803EBBB4F6A4440D7DCE2A2767B52C00905E9EB4E6A4440A8E6CF85767B52C0DC3E3A364F6A4440A9A9DE33767B52C0DCF83DEC4E6A4440DFADFDEC757B52C016A841AC4E6A4440C8FE0E9B757B52C08E3D49624E6A44406A417296757B52C0C8431E5E4E6A44404FA6DC81757B52C0A37CBA924E6A444047BCFE4A757B52C0F691F71E4F6A444016D95891757B52C032D67B5E4F6A4440B19BCADE757B52C0114268A44F6A44405A77A010767B52C0932E66D14F6A444057D1B278757B52C0CA4FBD55516A4440DF25D1C5747B52C047D4F91E536A44408D834FEC747B52C0A2D4B941536A4440963C33F7747B52C03D338F4B536A444010E12CFD747B52C076F14A3C536A4440')
    Right only
        ('2025040006', '0103000020E6100000020000006F000000963C33F7747B52C03D338F4B536A44409BBF46D2747B52C0E4D311AE536A44401797260A747B52C00388F7C3556A444038C928F5737B52C04E93D4B5556A444058216CA9727B52C0C8C279D6546A44408931EA73717B52C0BEF91406546A44405E5B5981717B52C0CC4EBEE3536A444054378EAD717B52C0700EBF72536A4440181360DB717B52C08164A2FD526A44409D0AA5FF717B52C0A5BEECA0526A44405B4EAB1C727B52C0FA58BE56526A4440E8BEAD39727B52C000AE970C526A4440E686133F727B52C0E955CCFE516A4440C17ABF56727B52C0A15A4AC2516A44407E13E127737B52C07A79BBAB4F6A4440E9F462DF737B52C01719AED64D6A44409E8DFAF2737B52C09F6E5FE84D6A444011F15947747B52C0D6638B344E6A44408967A099747B52C0BA3AD37E4E6A44400E689AC4747B52C05E29A1A54E6A44405CEF0D10757B52C0DFEEC5E44D6A4440149E2FF5747B52C05BC683CC4D6A444076E0D8A1747B52C0F81444814D6A44409A22014B747B52C0F8E1DC324D6A4440BAAC7034747B52C0650C7E1E4D6A44402C43711F747B52C051CC2A544D6A444070CB4A16737B52C08A4AC6644C6A444045F67F2C737B52C083F0042C4C6A4440EBA71F08737B52C0C2E72C0B4C6A444058402749737B52C0E4CBF6644B6A44403B09CD36737B52C0FAEE4B504B6A444079268626737B52C01419F0374B6A44405AFFA318737B52C03C275F1C4B6A444034CA6D0D737B52C0C87C22FE4A6A444031E91805737B52C08900D1DD4A6A4440EA8DD0FF727B52C0A4300DBC4A6A4440D15CAFFD727B52C00F537F994A6A4440426DBFFE727B52C0A086D3764A6A4440EC8FFB02737B52C072D4B7544A6A44402E4F4F0A737B52C0AC51D8334A6A444035649414737B52C0A151D9144A6A4440E3149921737B52C0CF8953F8496A44400FD61931737B52C0B0FED5DE496A444026C1CC42737B52C07829E2C8496A4440FB1F5656737B52C00F27E6B6496A4440BE6D566B737B52C0BADE38A9496A444022B36281737B52C0A2BB21A0496A444086700D98737B52C06315CD9B496A44400458E5AE737B52C0DB0D509C496A4440EDD877C5737B52C0BA91A8A1496A444094F154DB737B52C0175ABDAB496A4440CDE90DF0737B52C09B0D5ABA496A4440722A3931747B52C04226C713496A4440D9309855747B52C0C52B9F34496A444084854469747B52C014475602496A44404D910851757B52C0BEEB97D3496A4440CDEB6A72757B52C0BCB0BAF1496A4440284E3467757B52C020EC610E4A6A4440F9ED55E5757B52C084FB42804A6A44406330EE58767B52C0A5ECA1E84A6A4440FC41B1BF767B52C0314D68454B6A4440BA9B620D777B52C06DDB8C8B4B6A44407A414C4B777B52C0D91973C34B6A44408FBC5378777B52C0906D1BEC4B6A444015EEE349777B52C0E410CE624C6A4440C4762C32787B52C0D7A685344D6A4440EA1D3BA6787B52C085064F9D4D6A4440CBEFCEAF787B52C0975DF4A54D6A4440D0FED42B797B52C0AADDF2684C6A44404492F796787B52C0AD618CE24B6A4440B369B55C797B52C06B4B1AE9496A4440F8A34A88797B52C06B9873104A6A4440CCB269BD7A7B52C0C1EF8B274B6A4440C09456E17A7B52C0E758B8CB4A6A4440D22DB1E27A7B52C08B8B1FC94A6A444010A475EF7A7B52C0E0D7AFB04A6A4440AD15FBFF7A7B52C0A5C300994A6A44404F038F127B7B52C0E2C326854A6A444019A0D0267B7B52C02A278A754A6A4440F5F2543C7B7B52C009F37D6A4A6A4440F5EEAB527B7B52C011083C644A6A4440462D5F697B7B52C0DF31E3624A6A444046BEF57F7B7B52C0D4F57C664A6A44400ACEFB957B7B52C022EAF36E4A6A44408BFFFAAA7B7B52C07A4D1F7C4A6A444002E385BE7B7B52C0D170B78D4A6A4440046935D07B7B52C03B615FA34A6A4440266EABDF7B7B52C042E9A4BC4A6A4440DB1799EC7B7B52C0A07004D94A6A44400277B8F67B7B52C06FFAE8F74A6A4440F870D5FD7B7B52C0E116AE184B6A44407EECC9017C7B52C0888DA93A4B6A4440015E80027C7B52C0B97F275D4B6A44406C52F6FF7B7B52C0C725727F4B6A4440F1E239FA7B7B52C030BDD3A04B6A4440C5043BF67B7B52C08B4E38AF4B6A444073E265F17B7B52C0AB32A0C04B6A4440DB7843CD7B7B52C01808FD1C4C6A4440D3A3503D7C7B52C091BF26824C6A4440A6E892777B7B52C0E1FC987B4E6A44400FA6A1207B7B52C073C7182D4E6A4440ECB0C759797B52C0A1FFB9B7526A444031FBB8B0797B52C0B53F3A06536A4440037D8031797B52C028FF6A4B546A444017D903DF787B52C0DC67421E556A44404C185A99777B52C0AB533DF8536A444077429C31777B52C06E7A929A536A44400555C718777B52C0B52C0ADA536A4440107E90B3767B52C0A8DBC1DC546A4440AF4F6255757B52C0EAB396A0536A4440963C33F7747B52C03D338F4B536A44401900000010E12CFD747B52C076F14A3C536A4440621C692D767B52C0114BA532506A444021BAFB39767B52C08B128112506A4440D79B5A46767B52C0B540AB1D506A444077B6307C767B52C043B8484E506A44401439FB93767B52C095657911506A4440C3E78B6A777B52C08C7831D3506A4440B3A84EC3777B52C0A5BE4EF04F6A4440EFF44789777B52C07803EBBB4F6A4440D7DCE2A2767B52C00905E9EB4E6A4440A8E6CF85767B52C0DC3E3A364F6A4440A9A9DE33767B52C0DCF83DEC4E6A4440DFADFDEC757B52C016A841AC4E6A4440C8FE0E9B757B52C08E3D49624E6A44406A417296757B52C0C8431E5E4E6A44404FA6DC81757B52C0A37CBA924E6A444047BCFE4A757B52C0F691F71E4F6A444016D95891757B52C032D67B5E4F6A4440B19BCADE757B52C0114268A44F6A44405A77A010767B52C0932E66D14F6A444057D1B278757B52C0CA4FBD55516A4440DF25D1C5747B52C047D4F91E536A44408D834FEC747B52C0A2D4B941536A4440963C33F7747B52C03D338F4B536A444010E12CFD747B52C076F14A3C536A4440')
    Columns with diffs
        alt_arch_1
        arch_build
        bbl_int
        build_nme
        des_addres
        lm_orig
        notes
    Differences by column
        Notes
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       left                                                                                                                                                                                                                             right
            bbl        wkb_geometry                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
            1013150024 0106000020E61000000100000001030000000100000015000000E4823BC04B7E52C0372AED9F0460444083828F8C487E52C0F4A061EB01604440110F8271487E52C00465733502604440C0FEE07A477E52C0197B1265016044402EFEE84C447E52C0057558B5FE5F44406DE302CA407E52C0D6C9D9BDFB5F444073D804973E7E52C0D74A2AE2F95F444082D8EBB43F7E52C06A5358D3F65F4440D8DD65803E7E52C0F229ABCEF55F444034E149643F7E52C06B5AB05EF35F4440A223EE32407E52C0F7EFE828F15F4440B49BDD8C3F7E52C05F359B9CF05F4440568EADBA407E52C0F2BC3A62ED5F44405DAB5257427E52C05D8964F8E85F4440BDC336D7447E52C062C90D15EB5F44408BA6E62E4A7E52C05D3AA198EF5F4440F3FF6BC04D7E52C05523799CF25F4440C8575443537E52C09B958C44F75F444094F8F7A2517E52C02EE396B8FB5F444031E983C34D7E52C0DC044A5306604440E4823BC04B7E52C0372AED9F04604440  Not included in the designation is a 1959 addition by Harrison & Abramovitz, one of the more sensitive recent additions to an old New York building. \none of the more sensitive recent additions to an old New York building.  Not included in the designation is a 1959 addition by Harrison & Abramovitz, one of the more sensitive recent additions to an old New York building. \r\none of the more sensitive recent additions to an old New York building.
        Arch build
                                                                                                                                                                                                                                                                     left                           right
            bbl        wkb_geometry                                                                                                                                                                                                                                                                      
            1008300010 0106000020E61000000100000001030000000100000005000000AB36AB12577F52C0F8E2B5767F5F4440B06807A7597F52C084308D61785F44403C21AAC25A7F52C02CA38C50795F44407DD34C2E587F52C0F776B765805F4440AB36AB12577F52C0F8E2B5767F5F4440  Joseph Rogers\nJoseph Rogers  Joseph Rogers\r\nJoseph Rogers
        Des addres
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              left                                                                                    right
            bbl        wkb_geometry                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
            1005890048 0106000020E6100000010000000103000000010000000900000085CA5CF3278052C0ED5FB64E845D4440D1FFA578278052C03CABE59F825D44404156524A278052C0BFCC44FD815D44408F7545A6268052C084C055BD7F5D44407C29D5F1278052C03B20B9E27E5D44406788CE05288052C088508CD57E5D44409800986A2A8052C0E148CE3C875D444061600D0B298052C0E4E19924885D444085CA5CF3278052C0ED5FB64E845D4440                                                                                                                                                                                                                                                                  233 Bleecker Street (aka 233-237 Bleecker Street; 15 Carmine Street)\nCarmine Street)  233 Bleecker Street (aka 233-237 Bleecker Street; 15 Carmine Street)\r\nCarmine Street)
            1008980047 0106000020E61000000100000001030000000100000005000000CC4486CC017F52C064A4F9491E5E4440E6F5153F037F52C0FFCDEA511A5E44400D052D96047F52C0FF2684731B5E44403D709D23037F52C08530916B1F5E4440CC4486CC017F52C064A4F9491E5E4440                                                                                                                                                                                                                                                                                                                                                                                                                                             216 East 18th Street\n216 East 18th Street                                             216 East 18th Street\r\n216 East 18th Street
            4080940018 0106000020E6100000010000000103000000010000001100000011C62DECEB6F52C040D34738D3624440E9643ED4EB6F52C0008A94C3D2624440C436EB74EB6F52C0337AC8F0D26244403675FD40EB6F52C0BDA172C7D26244400DF3A11FEB6F52C0C5C6D224D26244405207E23AEB6F52C0E922D8D5D16244407D32359AEB6F52C02D34A4A8D162444023359D96EB6F52C0C5851F97D162444028B89D11ED6F52C0DB7867E3D0624440CDD11F36ED6F52C018396895D1624440B6D0B3ACED6F52C08DD37CD7D3624440E59344AEED6F52C02EBC1FDFD3624440170510A7ED6F52C0BC8489E2D3624440D63AD62AEC6F52C0D619D796D4624440251AF5C5EB6F52C08B8DADC6D462444000F2DE7EEB6F52C0B7851D6CD362444011C62DECEB6F52C040D34738D3624440    38 Hillcrest Avenue (aka 238-02 38th Road; 38-37 238th Street)\n38 Hillcrest Avenue    38 Hillcrest Avenue (aka 238-02 38th Road; 38-37 238th Street)\r\n38 Hillcrest Avenue
            5044410001 0106000020E610000001000000010300000001000000050000009A1607A9498952C01E2563801F4944406F18F60A4B8952C04F4246221F494440E6AE184D4B8952C01A29CB6421494440489529EB498952C02E23E6C2214944409A1607A9498952C01E2563801F494440                                                                                                                                                                                                                                                                                                                                                                                                           441 Clarke Avenue, formerly 218 Center Street\nCourt Pl and St. Patrick’s Pl           441 Clarke Avenue, formerly 218 Center Street\r\nCourt Pl and St. Patrick’s Pl
            1008300010 0106000020E61000000100000001030000000100000005000000AB36AB12577F52C0F8E2B5767F5F4440B06807A7597F52C084308D61785F44403C21AAC25A7F52C02CA38C50795F44407DD34C2E587F52C0F776B765805F4440AB36AB12577F52C0F8E2B5767F5F4440                                                                                                                                                                                                                                                                                                                                                                                                                        49 West 28th Street \n49 West 28th Street \n49 West 28th Street                      49 West 28th Street \r\n49 West 28th Street \r\n49 West 28th Street
        Build nme
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               left                             right
            bbl        wkb_geometry                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
            1012520034 0106000020E6100000010000000103000000020000001C00000088B5E1AA787E52C0F40D85C7AD6544400200F44F787E52C09E96F879AD65444012AE6AA0777E52C0A31B43E4AC6544407A3252A2777E52C0CD9518DFAC65444015E821AA777E52C032C0F2C9AC654440B57D1DAE777E52C0EDB129BFAC6544403F6C93B1777E52C0B845C8B5AC654440E739F5CC777E52C0DE4F996BAC654440DFBE3FF3777E52C02762E003AC654440CF9240B9737E52C062671E69A8654440F98F07A4737E52C092A60457A86544408BCD7AA8737E52C0EBE6F24AA865444007A1D99C747E52C076A1F6B4A56544406F940CD0747E52C0C7A5432AA56544404F5A9853747E52C0BFE31FC0A465444027044E01757E52C014888CE9A265444014FD57B5757E52C00512D401A1654440B23238337A7E52C066907BD6A46544402487EB317A7E52C012EC2A69A6654440ACADC82F7A7E52C04906BBFFA86544406BA86A2C7A7E52C00DE5EE13AD6544408731FC2B7A7E52C0422A1B9AAD6544405CEBD5277A7E52C026B3C707AE654440FD38890B7A7E52C0B5A57D7FAE6544409A00F8FB797E52C09E2551C1AE65444041E28EF4797E52C0BC9AABE0AE654440BF69E930797E52C07031D239AE65444088B5E1AA787E52C0F40D85C7AD65444009000000C619D06F787E52C030FFED05A7654440C89707D4767E52C064E2BFA6A5654440117DFDF5757E52C0BFBE4000A8654440F39F3EEC757E52C0D95DA61AA865444012C0883C767E52C076401F5FA865444050A2220C777E52C0B0CD2A10A9654440BB360788777E52C04994D479A9654440CAD1DAA8777E52C06256E920A9654440C619D06F787E52C030FFED05A7654440  Stratford-Avon\nStratford-Avon  Stratford-Avon\r\nStratford-Avon
        Alt arch 1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               left                                     right
            bbl        wkb_geometry                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
            1008940013 0106000020E6100000010000000103000000010000000F00000013275F769A7E52C0FD8BEDC7ED5F44405C87585D9B7E52C0E5EB454EEB5F4440A478D18F9C7E52C069517E05E85F4440B38C40209D7E52C05DFE4179E65F44406811180D9E7E52C058D1FB40E75F44400EC9C62E9D7E52C0CC82E4A2E95F444085CA3C7C9C7E52C041ABB38CEB5F44409200B7F59B7E52C0CE03441BEB5F44402C3710A59B7E52C0D38987F8EB5F4440536E66749B7E52C03412097EEC5F4440E614948C9B7E52C090DC6C92EC5F444066D9DD5E9B7E52C0228ED70FED5F44401B3F79C89B7E52C02AEDE568ED5F4440F4977C5E9B7E52C06806AA8BEE5F444013275F769A7E52C0FD8BEDC7ED5F4440  George Henry Bissel\nGeorge Henry Gale  George Henry Bissel\r\nGeorge Henry Gale
        Bbl int
            37553 rows. First 20 shown
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   left         right
            bbl        wkb_geometry                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
            3066920018 0106000020E610000001000000010300000001000000050000000A8F6C8D717D52C08FBE676CC35044405A714CBA727D52C059D00A39C3504440AE639FD9727D52C06F0710E3C45044407272BFAC717D52C039E76E16C55044400A8F6C8D717D52C08FBE676CC3504440                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  3066920018  3066920018.0
            3066920074 0106000020E61000000100000001030000000100000005000000759642FD6F7D52C0F0F32AC1C0504440B3BFC71B717D52C01263F882C05044406A435F36717D52C045BD629FC1504440D210DA17707D52C0375095DDC1504440759642FD6F7D52C0F0F32AC1C0504440                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  3066920074  3066920074.0
            3052370230 0106000020E61000000100000001030000000100000009000000C89EC2478F7D52C0EDCB1EF0FE504440D21837398F7D52C09140C53EFE504440E302D1BB8E7D52C0D01FA656FE5044400B0F3F818E7D52C0CDE1908CFB504440E1831C638E7D52C0585A4D92FB504440E4CEE8388E7D52C06ED0BD8FF9504440290657FA8F7D52C0B932283AF9504440D8BCA96D907D52C0636828B8FE504440C89EC2478F7D52C0EDCB1EF0FE504440                                                                                                                                                                                                                                                                                                                                                                  3052370230  3052370230.0
            3066910032 0106000020E610000001000000010300000001000000090000004C0B9904887D52C0C78A3002B85044405F2C8C11897D52C02BED1FD1B75044402441085B897D52C09BAB5378BB504440FD337CE8877D52C0D0EEEDBBBB504440FF680EE3877D52C0AF55EABCBB5044403AC8A5DD877D52C011F51978BB504440BCEEA5A1877D52C096768D7CB850444093C3AD0C887D52C06E820569B85044404C0B9904887D52C0C78A3002B8504440                                                                                                                                                                                                                                                                                                                                                                  3066910032  3066910032.0
            3066930007 0106000020E6100000010000000103000000010000000D000000208905AE617D52C0F6537CBAB85044408F6ECBA4617D52C0E00D385CB850444081452D0E617D52C0C904707EB850444096CB3AA3607D52C0CE8A5D39B4504440E00E72C9607D52C0DFC8AE30B45044407C11F5B3607D52C09EE10F55B35044400652A9CD627D52C056D6E8DAB25044401B7D3604637D52C05C076F08B55044407AD4F5A7627D52C05B28641DB550444043B3D9F1627D52C0E2468F10B8504440BD20725B627D52C0DDCBB932B85044408FF8AA64627D52C03622FC90B8504440208905AE617D52C0F6537CBAB8504440                                                                                                                                                                                                                                  3066930007  3066930007.0
            3066920045 0106000020E6100000010000000103000000010000000F0000002A98853E797D52C03FC682D5F3504440AC4AD52D797D52C00F7E3120F3504440E485EBFE787D52C01422392AF35044400CF78F9C787D52C01C3191FDEE504440A4B2E9C2787D52C0C1F95DF5EE5044400AE7DAB3787D52C068ECC151EE5044407929C466797D52C0E438872BEE504440A74B70EB797D52C0AB942D0FEE50444046CAD1F9797D52C07DAC75ABEE504440AF387E687A7D52C0E5DFCE93EE504440A00393AB7A7D52C0A7D0A46CF1504440BB0A5A2A7B7D52C03A0C8D51F150444007C3F5387B7D52C044AD49F0F1504440F272FD5A7B7D52C0DD380462F35044402A98853E797D52C03FC682D5F3504440                                                                                                                                                                  3066920045  3066920045.0
            3066910048 0106000020E610000001000000010300000001000000090000008E6BFC357F7D52C03BBBFD14AE504440EC63C5BD7E7D52C0A7764431AE504440FED556C97B7D52C0C6D02FE3AE50444061E610AF7B7D52C0EE0AD8DFAD50444010D2B8137C7D52C014922BC8AD504440DB829BFC7B7D52C0016CFFE3AC504440AE68820D7E7D52C0545E9867AC50444064FE98047F7D52C06DC97B2DAC5044408E6BFC357F7D52C03BBBFD14AE504440                                                                                                                                                                                                                                                                                                                                                                  3066910048  3066910048.0
            3052380011 0106000020E61000000100000001030000000100000005000000BE693CEE777D52C0C3682D1706514440CAA47D13797D52C0CD8943F00551444007C4592D797D52C00620BAB407514440B6791808787D52C0F811A2DB07514440BE693CEE777D52C0C3682D1706514440                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  3052380011  3052380011.0
            3066940016 0106000020E610000001000000010300000001000000070000001FD1F57E527D52C09B0271B3C5504440D6DF7668537D52C0B7DAD076C550444098DE5D9E537D52C03B4ED658C7504440227892E3527D52C070AC5589C75044400A7D2D4F527D52C04DBEDBAFC75044408CD54719527D52C01746D6CDC55044401FD1F57E527D52C09B0271B3C5504440                                                                                                                                                                                                                                                                                                                                                                                                                                  3066940016  3066940016.0
            3052380011 0106000020E6100000010000000103000000010000000E0000006F0E39FB797D52C0099D387606514440A853AA2A7A7D52C02CEDD90C065144409B7A5BB77A7D52C014DEAFEB055144409C02E6DF7A7D52C00564B019065144409832D1E47A7D52C074D6284A06514440AF36BF377B7D52C032199B36065144402450D52B7B7D52C0F06046C105514440756EFB837C7D52C04C0F2770055144408DEB09DD7C7D52C04F0035DD085144403E9A7FE0797D52C01C4B6E910951444040EBBEB3797D52C0D8BAADD807514440F460FA067A7D52C073AA0EC5075144405A5480E5797D52C057D1577B065144406F0E39FB797D52C0099D387606514440                                                                                                                                                                                                  3052380011  3052380011.0
            3066920048 0106000020E610000001000000010300000001000000090000002FCC29C8767D52C08CD9D09AEF504440125E6717787D52C08702B75DEF504440E66C8078787D52C043DEAE32F45044404C214F44767D52C08E9B8199F45044409E4E9811767D52C043D06F13F2504440E802E24C767D52C06746A108F2504440D6055433767D52C0D6F020C3F0504440FE08FFDC767D52C0A3B234A4F05044402FCC29C8767D52C08CD9D09AEF504440                                                                                                                                                                                                                                                                                                                                                                  3066920048  3066920048.0
            3066930047 0106000020E61000000100000001030000000100000005000000EF22F85C637D52C0BBAD2102F15044406F3C1AAB647D52C02A0830CDF0504440E17D33CC647D52C0FAD61DB2F25044409E51117E637D52C08F7F0FE7F2504440EF22F85C637D52C0BBAD2102F1504440                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  3066930047  3066930047.0
            3066920015 0106000020E61000000100000001030000000100000009000000D992369E737D52C061996AB7BE504440C1029DF1757D52C05FF9DB45BE504440B9A4B637767D52C0E69E2A9BC1504440C5F91929737D52C075367030C250444020AA6101737D52C04FE7EC4CC0504440842E0086737D52C034B1A233C0504440DC001075737D52C080E57865BF504440ED70A7AB737D52C0A70F105BBF504440D992369E737D52C061996AB7BE504440                                                                                                                                                                                                                                                                                                                                                                  3066920015  3066920015.0
            3010650043 0106000020E6100000010000000103000000010000000C000000E209597C577E52C00C82FC162D564440BAF08E27577E52C092490D152E56444064434489567E52C07C735AEF2F564440A14F6A51567E52C0DE0DB6963056444032B55706557E52C0C04A80962F564440CADC0FAD547E52C049B068512F5644400DB2E1BE547E52C05C7F021C2F564440321EC134557E52C0BDDFD1BA2D56444031BDD282557E52C08FEAE5D02C564440DAD9F7E7557E52C06A8E2A1F2D564440FFD1233D567E52C006C6F71F2C564440E209597C577E52C00C82FC162D564440                                                                                                                                                                                                                                                                  3010650043  3010650043.0
            3010870032 0106000020E6100000010000000103000000010000000E00000032F55CB5777E52C084B3AE6754554440133DD13E797E52C08F9C21BD505544407FB15393797E52C0BA5F8AF34F5544403670409D797E52C02AC3DEDB4F554440E1E6D2A9797E52C0B60D19E84F5544409A330B197A7E52C02A443A5450554440EF922CE8797E52C0B105CCC850554440C42D25307A7E52C05108C70E51554440EDBB94417A7E52C00B9995B3515544405A3E99B6787E52C0BCADC8615555444026151F6A787E52C0B188EF95555544402F62CB09787E52C0A7194938555544401794E1FB777E52C08AF83FAC5455444032F55CB5777E52C084B3AE6754554440                                                                                                                                                                                                  3010870032  3010870032.0
            3011360055 0106000020E61000000100000001030000000100000005000000566F605B2C7E52C003FBD075145744402BCCD5AA2C7E52C077E298C5125744400670791F2F7E52C0BC9B86D11357444067EB02D02E7E52C095C1BE8115574440566F605B2C7E52C003FBD07514574440                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  3011360055  3011360055.0
            3013200031 0106000020E610000001000000010300000001000000140000005A229748E77C52C0303B7DD9DB5444400AFB342FE77C52C0306F2C03DA544440BA2D3515E77C52C0FF876F21D8544440E25EF440E87C52C074E5E8FBD754444048110160E87C52C096343F3BDA5444409BB21D74E87C52C07185DAAFDB5444404C2E6FFEE77C52C02C5895BEDB5444408C1784F7E77C52C026F075D7DB544440F310CDEDE77C52C0DDC611EEDB544440B5E598E1E77C52C00258AD01DC5444400A6A49D3E77C52C06325AA11DC544440860556C3E77C52C051B7861DDC54444044003FB2E77C52C03578E224DC544440850C90A0E77C52C021A47F27DC544440D8A7D88EE77C52C03A034B25DC5444407E91A97DE77C52C03E1C561EDC54444001B68F6DE77C52C06F33D712DC5444408A900C5FE77C52C0C2152F03DC544440E2B59852E77C52C08A7FDDEFDB5444405A229748E77C52C0303B7DD9DB544440  3013200031  3013200031.0
            3019290031 0106000020E61000000100000001030000000100000007000000AF31AD04F77D52C066202D582F584440902AB5BDF77D52C0B945262E2F58444089C6FCB9F97D52C0AE6AB7BA2E5844401371D0E7F97D52C0065C588E3058444016B78032F77D52C04F1ACE2B31584440082FE526F77D52C082C05DB530584440AF31AD04F77D52C066202D582F584440                                                                                                                                                                                                                                                                                                                                                                                                                                  3019290031  3019290031.0
            3010680025 0106000020E6100000010000000103000000010000000E0000003326A86B367E52C071209E9D0B564440D8CC89E0377E52C02D26C23F07564440ECAA574E387E52C0D32F93F605564440FBA89837397E52C02ED500AB065644403FAE1A5B397E52C094DD76C60656444079D39FD0387E52C0EF5D9B65085644406C4B6B78377E52C01C02806D0C56444062979358377E52C0459ADF540C5644406548753E377E52C063EAEA6D0C564440170C3436377E52C03028A9860C564440D2FD2DC9367E52C05BC053320C5644405F3A6FD1367E52C0D88295190C564440F9FB14C4367E52C0478302E20B5644403326A86B367E52C071209E9D0B564440                                                                                                                                                                                                  3010680025  3010680025.0
            3018390015 0106000020E6100000010000000103000000010000000D00000019412915857C52C08F0431066A5744402BDD38A6847C52C0A12F04B365574440FFC8E476847C52C0FE8AA3DA635744408EFF09A7847C52C0C9D6B78763574440B8C51B01857C52C056A1CB72635744408F414748857C52C0B3E001AA635744400A619B77857C52C0B082628265574440E1C02FC9857C52C03BD4706F65574440BD041F38867C52C01EB19BC269574440015ED815867C52C0334B75106A57444084475090857C52C06F4C782F6A57444052FCEA70857C52C0A3A5E0F06957444019412915857C52C08F0431066A574440                                                                                                                                                                                                                                  3018390015  3018390015.0
        Lm orig
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    left                                                                                                                          right
            bbl        wkb_geometry                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
            1010440022 0106000020E6100000010000000103000000010000000F0000007698D8930D7F52C09F8AB88AEE6144409F1D3E850D7F52C04A86657EEE61444065A8728C0D7F52C0E4DDAA6AEE6144405EB27906117F52C0B4619DE4E4614440D94F2A41117F52C0D3DFD743E46144407837A32C127F52C00B2B9C0AE56144407A0F72BC117F52C03419F03DE66144409495C14F127F52C0DC714ABAE6614440C37822C6127F52C096680376E561444068A4C10B137F52C0200E49B7E4614440C710FD10147F52C02269CC93E56144408C595B9A137F52C01D73C3D8E66144409F368F10107F52C059E5078AF0614440B63B4109107F52C0701A0A9EF06144407698D8930D7F52C09F8AB88AEE614440  Former Eleventh Judicial District Courthouse (Eleventh District\nMunicipal Court/Seventh District Magistrates' Court) (now*  Former Eleventh Judicial District Courthouse (Eleventh District\r\nMunicipal Court/Seventh District Magistrates' Court) (now*

Notes (same as lpc_scenic_landmarks ):

Left only column present: library used to ingest this geospatial dataset as a csv, resulting in 2 duplicate geom columns, the_geom & wkb_geometry. In new ingest template, we are extracting it as a geojson and it has one geom column. The duplicate geom column the_geom from library isn't used downstream. Therefore, we can ignore the diff.

Other Notes:

  • According to the report above, the following columns have differences:

        alt_arch_1
    arch_build
    bbl_int
    build_nme
    des_addres
    lm_orig
    notes
    • What's happening in these columns and why it's not corrected?
      It's not being corrected in ingest because 1) these columns are not used downstream and 2) the data diffs are neglectable. For all of these columns except bbl_int, the diff is in new line character: library uses \n and ingest uses \r\n. For bbl_int, library has bbl as a string integer (i.e. 1010440022); ingest has it as a string decimal (i.e. 1010440022.0)

  • There are 1 record in the left and 1 record in the right tables: safe to ignore. The error is due to geometries not being equal. However, they are geospatially equal:

    Image

    Image

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 4, 2024

fdny_firehouses

________________________________________________________________________________
Tables
    Left: fdny_firehouses_library
    Right: fdny_firehouses_ingest
________________________________________________________________________________
Row count
    Left: 219
    Right: 219
________________________________________________________________________________
Column comparison
    Both
        bbl
        bin
        borough
        census_tract
        community_board
        community_council
        data_library_version
        facilityaddress
        facilityname
        latitude
        longitude
        nta
        ogc_fid
        postcode
        wkb_geometry
    Left only: None
    Right only: None
    Type differences
        Community council
            Left: character varying
            Right: text
        Community board
            Left: character varying
            Right: text
        Borough
            Left: character varying
            Right: text
        Facilityname
            Left: character varying
            Right: text
        Postcode
            Left: character varying
            Right: text
        Census tract
            Left: character varying
            Right: text
        Bbl
            Left: character varying
            Right: text
        Nta
            Left: character varying
            Right: text
        Bin
            Left: character varying
            Right: text
        Facilityaddress
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Compared columns
        bbl
        bin
        borough
        census_tract
        community_board
        community_council
        facilityaddress
        facilityname
        latitude
        longitude
        nta
        postcode
        wkb_geometry
    Left only
        Empty DataFrame
        Columns: [community_council, community_board, longitude, latitude, facilityname, postcode, census_tract, bbl, nta, borough, wkb_geometry, bin, facilityaddress]
        Index: []
    Right only
        Empty DataFrame
        Columns: [community_council, community_board, longitude, latitude, facilityname, postcode, census_tract, bbl, nta, borough, wkb_geometry, bin, facilityaddress]
        Index: []

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 4, 2024

dsny_garages

________________________________________________________________________________
Tables
    Left: dsny_garages_library
    Right: dsny_garages_ingest
________________________________________________________________________________
Row count
    Left: 80
    Right: 80
________________________________________________________________________________
Column comparison
    Both
        address
        bbl
        bin
        boro
        borocd
        city
        data_library_version
        districtcode
        name
        objectid
        ogc_fid
        state
        type
        wkb_geometry
        zip
    Left only: None
    Right only: None
    Type differences
        Bin
            Left: character varying
            Right: text
        State
            Left: character varying
            Right: text
        Objectid
            Left: character varying
            Right: text
        Address
            Left: character varying
            Right: text
        Name
            Left: character varying
            Right: text
        Districtcode
            Left: character varying
            Right: text
        Bbl
            Left: character varying
            Right: text
        Borocd
            Left: character varying
            Right: text
        Type
            Left: character varying
            Right: text
        Zip
            Left: character varying
            Right: text
        Boro
            Left: character varying
            Right: text
        City
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Compared columns
        address
        bbl
        bin
        boro
        borocd
        city
        districtcode
        name
        objectid
        state
        type
        wkb_geometry
        zip
    Left only
        Empty DataFrame
        Columns: [bin, state, objectid, address, name, districtcode, bbl, type, wkb_geometry, borocd, zip, boro, city]
        Index: []
    Right only
        Empty DataFrame
        Columns: [bin, state, objectid, address, name, districtcode, bbl, type, wkb_geometry, borocd, zip, boro, city]
        Index: []

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 4, 2024

dsny_frequencies

________________________________________________________________________________
Tables
    Left: dsny_frequencies_library
    Right: dsny_frequencies_ingest
________________________________________________________________________________
Row count
    Left: 610
    Right: 610
________________________________________________________________________________
Column comparison
    Both
        data_library_version
        district
        freq_bulk
        freq_ewaste
        freq_organics
        freq_recycling
        freq_refuse
        freq_resfuseid
        frequency
        objectid
        ogc_fid
        schedulecode
        section
        shape_area
        shape_length
        wkb_geometry
    Left only
        multipolygon
    Right only: None
    Type differences
        District
            Left: character varying
            Right: text
        Freq recycling
            Left: character varying
            Right: text
        Freq bulk
            Left: character varying
            Right: text
        Section
            Left: character varying
            Right: text
        Freq resfuseid
            Left: character varying
            Right: text
        Freq organics
            Left: character varying
            Right: text
        Shape area
            Left: character varying
            Right: text
        Objectid
            Left: character varying
            Right: text
        Schedulecode
            Left: character varying
            Right: text
        Shape length
            Left: character varying
            Right: text
        Freq refuse
            Left: character varying
            Right: text
        Frequency
            Left: character varying
            Right: text
        Freq ewaste
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Compared columns
        district
        freq_bulk
        freq_ewaste
        freq_organics
        freq_recycling
        freq_refuse
        freq_resfuseid
        frequency
        objectid
        schedulecode
        section
        shape_area
        shape_length
        wkb_geometry
    Left only
        Empty DataFrame
        Columns: [district, wkb_geometry, freq_recycling, freq_bulk, section, freq_resfuseid, freq_organics, shape_area, shape_length, objectid, schedulecode, freq_refuse, frequency, freq_ewaste]
        Index: []
    Right only
        Empty DataFrame
        Columns: [district, wkb_geometry, freq_recycling, freq_bulk, section, freq_resfuseid, freq_organics, shape_area, shape_length, objectid, schedulecode, freq_refuse, frequency, freq_ewaste]
        Index: []

Notes:

  • Left only column present: library used to ingest this geospatial dataset as a csv, resulting in 2 duplicate geom columns, multipolygon & wkb_geometry. In new ingest template, we are extracting it as a geojson and it has one geom column. The duplicate geom column multipolygon from library isn't used downstream. Therefore, we can ignore the diff.

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 5, 2024

dpr_schoolyard_to_playgrounds

________________________________________________________________________________
Tables
    Left: dpr_schoolyard_to_playgrounds_library
    Right: dpr_schoolyard_to_playgrounds_ingest
________________________________________________________________________________
Row count
    Left: 289
    Right: 289
________________________________________________________________________________
Column comparison
    Both
        acres
        address
        borough
        communityboard
        councildistrict
        data_library_version
        dsfbldgcode
        gispropnum
        jurisdiction
        location
        nys_assembly
        nys_senate
        ogc_fid
        us_congress
        wkb_geometry
        zipcode
    Left only: None
    Right only: None
    Type differences
        Gispropnum
            Left: character varying
            Right: text
        Nys assembly
            Left: character varying
            Right: text
        Communityboard
            Left: character varying
            Right: text
        Address
            Left: character varying
            Right: text
        Councildistrict
            Left: character varying
            Right: text
        Location
            Left: character varying
            Right: text
        Nys senate
            Left: character varying
            Right: text
        Us congress
            Left: character varying
            Right: text
        Borough
            Left: character varying
            Right: text
        Dsfbldgcode
            Left: character varying
            Right: text
        Zipcode
            Left: character varying
            Right: text
        Jurisdiction
            Left: character varying
            Right: text
        Acres
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Compared columns
        acres
        address
        borough
        communityboard
        councildistrict
        dsfbldgcode
        gispropnum
        jurisdiction
        location
        nys_assembly
        nys_senate
        us_congress
        wkb_geometry
        zipcode
    Left only
        Empty DataFrame
        Columns: [gispropnum, address, communityboard, nys_assembly, councildistrict, location, nys_senate, us_congress, wkb_geometry, dsfbldgcode, zipcode, jurisdiction, borough, acres]
        Index: []
    Right only
        Empty DataFrame
        Columns: [gispropnum, address, communityboard, nys_assembly, councildistrict, location, nys_senate, us_congress, wkb_geometry, dsfbldgcode, zipcode, jurisdiction, borough, acres]
        Index: []

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 5, 2024

dpr_park_access_zone

________________________________________________________________________________
Tables
    Left: dpr_park_access_zone_library
    Right: dpr_park_access_zone_ingest
________________________________________________________________________________
Row count
    Left: 1
    Right: 1
________________________________________________________________________________
Column comparison
    Both
        data_library_version
        id
        ogc_fid
        type
        wkb_geometry
    Left only: None
    Right only: None
    Type differences
        Type
            Left: character varying
            Right: text
        Id
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Compared columns
        id
        type
        wkb_geometry
    Left only
        Empty DataFrame
        Columns: [wkb_geometry, type, id]
        Index: []
    Right only
        Empty DataFrame
        Columns: [wkb_geometry, type, id]
        Index: []

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 5, 2024

dpr_greenthumb

________________________________________________________________________________
Tables
    Left: dpr_greenthumb_library
    Right: dpr_greenthumb_ingest
________________________________________________________________________________
Row count
    Left: 629
    Right: 629
________________________________________________________________________________
Column comparison
    Both
        address
        assemblydist
        bbl
        borough
        censustract
        communityboard
        congressionaldist
        coundist
        crossstreets
        data_library_version
        gardenname
        juris
        lat
        lon
        nta
        ogc_fid
        openhrsf
        openhrsm
        openhrssa
        openhrssu
        openhrsth
        openhrstu
        openhrsw
        parksid
        policeprecinct
        statesenatedist
        status
        wkb_geometry
        zipcode
    Left only: None
    Right only: None
    Type differences
        Policeprecinct
            Left: character varying
            Right: text
        Parksid
            Left: character varying
            Right: text
        Zipcode
            Left: character varying
            Right: text
        Lat
            Left: character varying
            Right: text
        Gardenname
            Left: character varying
            Right: text
        Assemblydist
            Left: character varying
            Right: text
        Address
            Left: character varying
            Right: text
        Openhrsw
            Left: character varying
            Right: text
        Bbl
            Left: character varying
            Right: text
        Nta
            Left: character varying
            Right: text
        Status
            Left: character varying
            Right: text
        Censustract
            Left: character varying
            Right: text
        Openhrsf
            Left: character varying
            Right: text
        Openhrssa
            Left: character varying
            Right: text
        Juris
            Left: character varying
            Right: text
        Openhrssu
            Left: character varying
            Right: text
        Crossstreets
            Left: character varying
            Right: text
        Lon
            Left: character varying
            Right: text
        Openhrstu
            Left: character varying
            Right: text
        Coundist
            Left: character varying
            Right: text
        Borough
            Left: character varying
            Right: text
        Communityboard
            Left: character varying
            Right: text
        Congressionaldist
            Left: character varying
            Right: text
        Statesenatedist
            Left: character varying
            Right: text
        Openhrsth
            Left: character varying
            Right: text
        Openhrsm
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Compared columns
        address
        assemblydist
        bbl
        borough
        censustract
        communityboard
        congressionaldist
        coundist
        crossstreets
        gardenname
        juris
        lat
        lon
        nta
        openhrsf
        openhrsm
        openhrssa
        openhrssu
        openhrsth
        openhrstu
        openhrsw
        parksid
        policeprecinct
        statesenatedist
        status
        wkb_geometry
        zipcode
    Left only
        Empty DataFrame
        Columns: [policeprecinct, parksid, zipcode, lat, gardenname, assemblydist, address, wkb_geometry, openhrsw, bbl, nta, status, censustract, openhrsf, openhrssa, juris, openhrssu, crossstreets, lon, openhrstu, coundist, borough, communityboard, congressionaldist, statesenatedist, openhrsth, openhrsm]
        Index: []
    Right only
        Empty DataFrame
        Columns: [policeprecinct, parksid, zipcode, lat, gardenname, assemblydist, address, wkb_geometry, openhrsw, bbl, nta, status, censustract, openhrsf, openhrssa, juris, openhrssu, crossstreets, lon, openhrstu, coundist, borough, communityboard, congressionaldist, statesenatedist, openhrsth, openhrsm]
        Index: []

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 9, 2024

dpr_parksproperties

________________________________________________________________________________
Tables
    Left: dpr_parksproperties_library
    Right: dpr_parksproperties_ingest
________________________________________________________________________________
Row count
    Left: 2051
    Right: 2051
________________________________________________________________________________
Column comparison
    Both
        acquisitiondate
        acres
        address
        borough
        class
        communityboard
        councildistrict
        data_library_version
        department
        eapply
        gisobjid
        gispropnum
        globalid
        jurisdiction
        location
        mapped
        name311
        nys_assembly
        nys_senate
        objectid
        ogc_fid
        omppropid
        parentid
        permit
        permitdistrict
        permitparent
        pip_ratable
        precinct
        retired
        signname
        subcategory
        typecategory
        us_congress
        waterfront
        wkb_geometry
        zipcode
    Left only: None
    Right only: None
    Type differences
        Globalid
            Left: character varying
            Right: text
        Nys assembly
            Left: character varying
            Right: text
        Councildistrict
            Left: character varying
            Right: text
        Address
            Left: character varying
            Right: text
        Nys senate
            Left: character varying
            Right: text
        Retired
            Left: character varying
            Right: text
        Zipcode
            Left: character varying
            Right: text
        Parentid
            Left: character varying
            Right: text
        Omppropid
            Left: character varying
            Right: text
        Signname
            Left: character varying
            Right: text
        Typecategory
            Left: character varying
            Right: text
        Gispropnum
            Left: character varying
            Right: text
        Acres
            Left: character varying
            Right: text
        Borough
            Left: character varying
            Right: text
        Waterfront
            Left: character varying
            Right: text
        Us congress
            Left: character varying
            Right: text
        Location
            Left: character varying
            Right: text
        Permitparent
            Left: character varying
            Right: text
        Permitdistrict
            Left: character varying
            Right: text
        Eapply
            Left: character varying
            Right: text
        Pip ratable
            Left: character varying
            Right: text
        Permit
            Left: character varying
            Right: text
        Class
            Left: character varying
            Right: text
        Mapped
            Left: character varying
            Right: text
        Jurisdiction
            Left: character varying
            Right: text
        Department
            Left: character varying
            Right: text
        Gisobjid
            Left: character varying
            Right: text
        Precinct
            Left: character varying
            Right: text
        Acquisitiondate
            Left: timestamp with time zone
            Right: timestamp without time zone
        Name311
            Left: character varying
            Right: text
        Objectid
            Left: character varying
            Right: text
        Communityboard
            Left: character varying
            Right: text
        Subcategory
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Key columns
        gispropnum
        name311
    Left only: None
    Right only: None
    Columns with diffs: None
    Differences by column: None

Notes:

  • Ignoring the diff in type for Acquisitiondate column: this column isn't used downstream. Adding a processing step seems wasteful.

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 9, 2024

dpr_forever_wild

________________________________________________________________________________
Tables
    Left: dpr_forever_wild_library
    Right: dpr_forever_wild_ingest
________________________________________________________________________________
Row count
    Left: 139
    Right: 139
________________________________________________________________________________
Column comparison
    Both
        acres
        data_library_version
        forverwild
        gispropnum
        ogc_fid
        propertyna
        wkb_geometry
    Left only: None
    Right only: None
    Type differences
        Forverwild
            Left: character varying
            Right: text
        Propertyna
            Left: character varying
            Right: text
        Gispropnum
            Left: character varying
            Right: text
________________________________________________________________________________
Data comparison
    Key columns
        gispropnum
    Left only: None
    Right only: None
    Columns with diffs: None
    Differences by column: None

@sf-dcp
Copy link
Contributor

sf-dcp commented Dec 9, 2024

doitt_buildingfootprints_historical

________________________________________________________________________________
Tables
    Left: doitt_buildingfootprints_historical_library
    Right: doitt_buildingfootprints_historical_ingest
________________________________________________________________________________
Row count
    Left: 47351
    Right: 47351
________________________________________________________________________________
Column comparison
    Both
        base_bbl
        bin
        cnstrct_yr
        data_library_version
        demol_yr
        doitt_id
        feat_code
        geomsource
        globalid
        groundelev
        heightroof
        lstmoddate
        lststatype
        mpluto_bbl
        name
        ogc_fid
        shape_area
        shape_len
        wkb_geometry
    Left only: None
    Right only: None
    Type differences
        Feat code
            Left: character varying
            Right: text
        Shape len
            Left: character varying
            Right: text
        Lststatype
            Left: character varying
            Right: text
        Name
            Left: character varying
            Right: text
        Bin
            Left: character varying
            Right: text
        Shape area
            Left: character varying
            Right: text
        Mpluto bbl
            Left: character varying
            Right: text
        Globalid
            Left: character varying
            Right: text
        Geomsource
            Left: character varying
            Right: text
        Groundelev
            Left: character varying
            Right: text
        Cnstrct yr
            Left: character varying
            Right: text
        Doitt id
            Left: character varying
            Right: text
        Demol yr
            Left: character varying
            Right: text
        Base bbl
            Left: character varying
            Right: text
        Heightroof
            Left: character varying
            Right: text
        Lstmoddate
            Left: timestamp with time zone
            Right: timestamp without time zone
________________________________________________________________________________
Data comparison
    Compared columns
        base_bbl
        bin
        cnstrct_yr
        demol_yr
        doitt_id
        feat_code
        geomsource
        globalid
        groundelev
        heightroof
        lstmoddate
        lststatype
        mpluto_bbl
        name
        shape_area
        shape_len
        wkb_geometry
    Left only
        Empty DataFrame
        Columns: [mpluto_bbl, globalid, doitt_id, demol_yr, feat_code, geomsource, shape_len, base_bbl, heightroof, groundelev, shape_area, wkb_geometry, cnstrct_yr, lstmoddate, lststatype, name, bin]
        Index: []
    Right only
        Empty DataFrame
        Columns: [mpluto_bbl, globalid, doitt_id, demol_yr, feat_code, geomsource, shape_len, base_bbl, heightroof, groundelev, shape_area, wkb_geometry, cnstrct_yr, lstmoddate, lststatype, name, bin]
        Index: []

Notes:

  • Ignoring the diff in type for Lstmoddate column: this column isn't used downstream. Adding a processing step seems wasteful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sf-dcp sf-dcp

Labels
None yet
Projects
Data Engineering
Status: 🏗 In progress
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Library to Ingest: weekly OpenData templates NYCPlanning/data-engineering
2 participants
@fvankrieken @sf-dcp