Correct term end dates in historical terms, per #7

[JoshData]

This corrects term end dates in historical terms. Per #7, we wanted to fix term end dates to be the Constitutionally-defined end of a term.

In summary, this commit does a simple string substitution to correct the following dates:

  2nd Congress 1793-03-02 => 1793-03-03
 22nd Congress 1833-03-02 => 1833-03-03
 73rd Congress 1934-06-18 => 1935-01-03
 74th Congress 1936-06-20 => 1937-01-03
 75th Congress 1938-06-16 => 1939-01-03
 77th Congress 1942-12-16 => 1943-01-03
 78th Congress 1944-12-19 => 1945-01-03
 79th Congress 1946-08-02 => 1947-01-03
 80th Congress 1948-12-31 => 1949-01-03
 81st Congress 1951-01-02 => 1951-01-03
 82nd Congress 1952-07-07 => 1953-01-03
 83rd Congress 1954-12-02 => 1955-01-03
 84th Congress 1956-07-27 => 1957-01-03
 85th Congress 1958-08-24 => 1959-01-03
 86th Congress 1960-09-01 => 1961-01-03
 87th Congress 1962-10-13 => 1963-01-03
 88th Congress 1964-10-03 => 1965-01-03
 89th Congress 1966-10-22 => 1967-01-03
 90th Congress 1968-10-14 => 1969-01-03
 91st Congress 1971-01-02 => 1971-01-03
 92nd Congress 1972-10-18 => 1973-01-03
 93rd Congress 1974-12-20 => 1975-01-03
 94th Congress 1976-10-01 => 1977-01-03
 95th Congress 1978-10-15 => 1979-01-03
 96th Congress 1980-12-16 => 1981-01-03
 97th Congress 1982-12-23 => 1983-01-03
 98th Congress 1984-10-12 => 1985-01-03
 99th Congress 1986-10-18 => 1987-01-03
100th Congress 1988-10-22 => 1989-01-03
101th Congress 1990-10-28 => 1991-01-03
102th Congress 1992-10-09 => 1993-01-03
103th Congress 1994-12-01 => 1995-01-03
104th Congress 1996-10-04 => 1997-01-03
105th Congress 1998-12-19 => 1999-01-03
106th Congress 2000-12-15 => 2001-01-03
107th Congress 2002-11-22 => 2003-01-03
108th Congress 2004-12-09 => 2005-01-03
109th Congress 2006-12-09 => 2007-01-03
111th Congress 2010-12-22 => 2011-01-03

When I first created this dataset back in 2009, I used the sine die adjournment date of Congress as term end dates (as best as I could quickly identify those dates, and the later of the House or Senate). It has been a bit of a research project to figure out the actual term end dates, especially prior to 1933. Constitutional term end dates are more appropriate since a Member is a Member, and can come back into session, until the term actually ends, not when Congress last meets. In addition, I have found that historical roll call data from Voteview and historical legislative data from THOMAS/Congress.gov indicates there was legislative activity beyond the end dates that I have used --- so they may be incorrect anyway.

(Note that this doesn't address term start dates, which we've been storing as swearing-in dates, where known.)

According to the Constitution Annotated, Amendment 20, quoting the Senate report on the Twentieth Amendment (1 S. Rep. No. 26, 72d Cong., 1st Sess. 2, 4, 5, 6 (1932)):

The commencement of the terms of the first President and Vice President and of Senators and Representatives composing the First Congress was fixed by an act of [the Continental] Congress adopted September 13, 1788, and that act provided ‘that the first Wednesday in March next to be the time for commencing proceedings under the Constitution.’ It happened that the first Wednesday in March was the 4th day of March, and hence the terms of the President and Vice President and Members of Congress began on the 4th day of March.

This explains the frequency of March 3 as adjournment dates in our term end data through 1933 (72nd Congress), with only anomalies for the 2nd Congress (1793-03-02), 22nd Congress (1833-03-02), and 69th Congress (1927-03-04). I've verified that the 69th Congress did end on a March 4, and per further analysis in the Constitution Annotated (see below), it seems terms ended at some time on March 4. Nevertheless, I've left the March 3 end dates where they are (a total of 69 Congresses had that date), and I've left the 69th Congress's March 4 date alone. I've corrected the 2nd Congress and 22nd Congresses to March 3's (for consistency with the other Congresses in that time period).

The 20th Amendment changed the start and end dates of legislative (and executive) terms. The amendment was ratified on January 23, 1933. Per section 5 of the amendment, section 1 takes effect "on the 15th day of October following the ratification of this article", i.e. October 15, 1933. Section 1 states that legislative terms start and end "at noon on the 3d day of January, of the years in which such terms would have ended if this article had not been ratified." The 73rd Congress was in session. We have its adjournment date as 1934-06-18. But its actual end date was changed by the 20th Amendment from 1935-03-03 (or -04) to 1935-01-03 at noon. So this commit changes the term end date from 1934-06-18 to 1935-01-03.

The Constitution Annotated clarifies:

[The 20th Amendment] shortened, by the intervals between January 3 and March 4, the terms of Senators elected for terms ending March 4, 1935, 1937, and 1939 . . . It also shortened the terms of Representatives elected to the Seventy-third Congress, by the interval between January 3 and March 4, 1935.

Only a handful of Congresses since then have actually adjourned on January 3 (76th, 110th, 112th Congresses), so most of the term end dates starting with the 73rd Congress are updated by this commit to January 3.

Beginning with the 113th Congress we've been using January 3, so the last Congress updated here is the 111th Congress.

I made these changes using sed. Anywhere the date occurs in an end: is assumed to be the end of the term at the end of a Congress. Any ends of terms on other dates not listed above are left alone because they're resignations, deaths, etc.

[konklone]

This is a great piece of data validation work, thank you!

I made these changes using sed. Anywhere the date occurs in an end: is assumed to be the end of the term at the end of a Congress. Any ends of terms on other dates not listed above are left alone because they're resignations, deaths, etc.

Sounds safe enough to me. It's unlikely anyone would have resigned on the adjournment day of the closing session, and if someone happened to die on that last day, that error is clearly outweighed by the data improvement here.

If someone were concerned about it, there's probably a list of deaths (parsed from bioguide?) that we could compare to see if any death dates match the substitution list you've placed above.

[JoshData]

I was just thinking about how bioguide handles March 3/March 4. In the text of the bioguide (via #304), when a Member's service ends in the 69th Congress, they say March 3, 1927 --- even though Congress seemed to be in session on March 4. Conceivably March 4 was a continuation of the March 3 legislative day. So the dates I'm proposing to use don't line up with bioguide for the 69th Congress but should line up in all other Congresses.

if someone happened to die on that last day, that error is clearly outweighed by the data improvement here

Yeah and if we had it correct before, it was likely by accident!

[JoshData]

Last call? @dwillis, ok for you? This is a big commit so I don't want to catch anyone off guard.

[dwillis]

Ok by me. Nice work!

On Wednesday, September 2, 2015, Joshua Tauberer notifications@github.com
wrote:

Last call? @dwillis https://github.com/dwillis, ok for you? This is a
big commit so I don't want to catch anyone off guard.

—
Reply to this email directly or view it on GitHub
#305 (comment)
.