2OF1 · the depletionmode zineLinkedIn · X
← back to the issue

14 November 2010

Pulling “On this day in history” data from Wikipedia

Originally published on the old depletionmode / 2of1 blog (archived copy).

I've been looking around for an easy way to pull event information for a specific day from wikipedia in order to populate an internal database for one of my apps. Turns out it's dead simple. Something along the lines of the following code should do the trick (using the Beautiful Soup module): #!/usr/bin/env python2.6 from BeautifulSoup import BeautifulSoup as BSoup import urllib2 import re def remove_html_tags(html): p = re.compile(r' 0: print s except: pass Also as you can tell my Python skills leave much to be desired - but you get the point! And running: $ ./on-this-day.py 1685 – James II of England and VII of Scotland becomes King upon the death of his brother Charles II. 1778 – American Revolutionary War: In Paris the Treaty of Alliance and the Treaty of Amity and Commerce are signed by the United States and France signaling official recognition of the new republic. 1788 – Massachusetts becomes the sixth state to ratify the United State

I’ve been looking around for an easy way to pull event information for a specific day from wikipedia in order to populate an internal database for one of my apps.

Turns out it’s dead simple.

Something along the lines of the following code should do the trick (using the Beautiful Soup module):

listing text
#!/usr/bin/env python2.6
 
from BeautifulSoup import BeautifulSoup as BSoup
import urllib2
import re
 
def remove_html_tags(html):
    p = re.compile(r'<[^<]*?/?>')
    return p.sub('', html)
 
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
 
# Modify this for the date you're interested in
html = opener.open('http://en.wikipedia.org/wiki/February_6').read()
soup = BSoup(html)
for li in soup.html.body.findAll('ul')[1]:
    s = remove_html_tags(str(li))
    try:
        if int(s[0]) > 0:
            print s
    except:
        pass

Also as you can tell my Python skills leave much to be desired – but you get the point!

And running:

listing text
$ ./on-this-day.py
1685 – James II of England and VII of Scotland becomes King upon the death of his brother Charles II.
1778 – American Revolutionary War: In Paris the Treaty of Alliance and the Treaty of Amity and Commerce are signed by the United States and France signaling official recognition of the new republic.
1788 – Massachusetts becomes the sixth state to ratify the United States Constitution.
1806 – Battle of San Domingo British naval victory against the French in the Caribbean.
1815 – New Jersey grants the first American railroad charter to John Stevens.
1817 – The Argentinian San Martín crosses the Andes with an army in order to liberate Chile from Spanish rule.
1819 – Sir Thomas Stamford Raffles founds Singapore.
1820 – The first 86 African American immigrants sponsored by the American Colonization Society started a settlement in present-day Liberia.
1840 – Signing of the Treaty of Waitangi, establishing New Zealand as a British colony.
1843 – The first minstrel show in the United States, The Virginia Minstrels, opens (Bowery Amphitheatre in New York City).
1862 – American Civil War: Ulysses S. Grant gives the United States its first victory of the war, by capturing Fort Henry, Tennessee, known as the Battle of Fort Henry.
1899 – Spanish-American War: The Treaty of Paris, a peace treaty between the United States and Spain, is ratified by the United States Senate.
1900 – The international arbitration court at The Hague is created when the Netherlands' Senate ratifies an 1899 peace conference decree.
1922 – The Washington Naval Treaty is signed in Washington, D.C., limiting the naval armaments of United States, Britain, Japan, France, and Italy.
1933 – The 20th Amendment to the United States Constitution, establishing the beginning and ending of the terms of the elected federal offices, goes into effect.
1934 – Far right leagues rally in front of the Palais Bourbon in an attempted coup against the French Third Republic, creating a political crisis in France.
1942 – World War II: The United Kingdom declares war on Thailand.
1951 – The Broker, a Pennsylvania Railroad passenger train derails near Woodbridge Township, New Jersey. The accident kills 85 people and injures over 500 more. The wreck is one of the worst rail disasters in American history.
1952 – Elizabeth II becomes Queen upon the death of her father George VI. At the exact moment of succession, she was in a treehouse at the Treetops Hotel in Kenya.
1958 – Eight Manchester United F.C. players are killed in the Munich air disaster.
1959 – Jack Kilby of Texas Instruments files the first patent for an integrated circuit.
1959 – At Cape Canaveral, Florida, the first successful test firing of a Titan intercontinental ballistic missile is accomplished.
1978 – The Blizzard of 1978, one of the worst Nor'easters in New England history, hit the region, with sustained winds of 65&#160;mph and snowfall of 4" an hour.
1987 – Justice Mary Gaudron is appointed to the High Court of Australia, the first woman to be appointed.
1989 – The Round Table Talks start in Poland, thus marking the beginning of overthrow of communism in Eastern Europe.
1998 – Washington National Airport is renamed Ronald Reagan National Airport.
1998 – In Corsica, the prefect Claude Erignac is assassinated in Ajaccio, presumably by Yvan Colonna.

→ end of entry

2/1
@depletionmode
security research & systems engineering