Peter Doorn & Luuk Schreven [email protected] & [email protected]
description
Transcript of Peter Doorn & Luuk Schreven [email protected] & [email protected]
![Page 1: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/1.jpg)
Explorations of multi-level methods & ecological
inference techniques in the analysis of “Life Courses in
Context”
Peter Doorn & Luuk [email protected] & [email protected]
Data Archiving & Networked Services (DANS) Netherlands Institute for Scientific Information Services (NIWI)
![Page 2: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/2.jpg)
Structure of presentation
1. Introduction to “Life Courses in Context”-project
– Life Courses: Historical Sample of the Netherlands (HSN)
– Context: Census digitization project
2. Exploration of multi-level methods & ecological inference techniques
![Page 3: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/3.jpg)
Introduction to Life Courses in Context project
Two separate components:– Life Courses: Historical Sample of
the Netherlands (HSN)– Context: Digitization of (aggregate)
Census data 1795 – 1971
One combined grant application to Netherlands Organisation for Scientific Research (+ € 3.6 mln funding)
![Page 4: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/4.jpg)
Aim of Life Courses in Context project
‘…to develop a collaboratory for the study of 19th and 20th century population history.’
By combining the HSN and Census data sources:– HSN: Micro data + 40,000 individual
life courses– Census: aggregate data from
published census tables
![Page 5: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/5.jpg)
Life Courses: Historical Sample of the Netherlands
‘…to construct life courses as completely as possible for a representative portion of the 19th and 20th century population in the Netherlands.’
A sample has been drawn from the birth registers of all persons born in the Netherlands between 1812 and 1922 (sample size = 77,000 persons)
Data gathering by International Institute of Social History (IISH-IISG) since 1991
![Page 6: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/6.jpg)
Sources of HSN• (already mentioned) Birth registers basis of
samplePerson born, names, addresses, ages and
occupations of parents (literacy of father)• Death certificates
Place of residence, age and occupation of deceased, information on his/her spouse. In case of child occupation and literacy of father
• Marriage certificatesOccupations, place of residence and literacy of couple,
parents and witnesses• Dynamic population registration system (in use
since 1850) & personal record cards (later stage)Family structure, pattern of migration
• Land registers & tax records (later stage)Occupational history and wealth of subject
![Page 7: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/7.jpg)
Use of HSN1. Basic resource for historical research
in demography, sociology, epidemiology, socio-economics and social geography
2. Control database to compare research data
3. Foundation for the collection of new data
4. Source of expertise on data collecting
Questions? Contact Kees Mandemakers ([email protected])
![Page 8: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/8.jpg)
Context: Census Digitization 1795-1971
• ‘…to digitize all published (aggregate) census data from Dutch population, housing and occupational censuses between 1795 to 1971’
• National population censuses are one of the fundamental sources of information on conditions in a country, used in historical and social science research
• Information on population size and structural characteristics: age, gender, marital status, religion, household status, occupational activity and nationality
![Page 9: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/9.jpg)
Main objectives of Dutch censuses
1. To determine the size of the population on a fixed point in time
2. To probe and improve the reliability of the Dutch population registers
3. To examine the demographic and social-economic characteristics of the population
4. To provide data to facilitate domestic policy making
![Page 10: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/10.jpg)
Census Digitization projects: 1997 – present
• 1997 - 1999:– Scanning 200 books, 42.500 pages– Data-entry census 1899
• 2002 - March 2004:– Validation and correction of census data
1795-1859 and 1930– Digital archiving census 1960 and 1971
• March 2003 – December 2005:– Life Courses in Context (see: http://
www.lifecoursesincontext.nl)– Data-entry of census data 1869-1956– Documentation, harmonization, access and
research
![Page 11: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/11.jpg)
What has been realized?• New website up and running
– Only in Dutch! – Some 40,000 pages of tabular (aggregate)
census data downloadable from website– Documentation is available– Validation and correction are partially
complete– Harmonization schemes for certain census
variables• (restricted) Access to original micro
data files for 1960 and 1971 census– Van Tubergen & Maas (2005)
![Page 12: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/12.jpg)
Still to do…• Finishing validation and correction
• Building harmonization schemes for census variables:– HISCO for harmonization of occupations– Standardizing sub municipal divisions – Harmonizing other variables and categories
• Better access to the data– Data not only as Excel spreadsheets– StatLine or Nesstar? Or other publication tool?
• Translation of the website to English!
![Page 13: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/13.jpg)
Combining HSN & Census datasets
• Census covers whole population; check on data collected in sample
• Data sets are complementary; more data will be available
• HSN data are longitudinal; census data are cross-sectional snapshots
• Census data provide more regional detail
• Combining data can result in identification of individuals (privacy issues!)
![Page 14: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/14.jpg)
Comparison of variablesHSN micro data (birth, death and marriage registers)
Census aggregate data
Date (and hour) of birth Age (groups)
Place of birth (municipality of birth certificate) Municipality of birth (nationality, ethnicity)
Sex Sex
Date of marriage NA
Place of marriage NA
Marital status Marital status
Occupational title Occupation (-al group, sector)
NA Religion
Address Neighborhood/municipality
NA Characteristics of dwelling (housing censuses)
Age of parents at birth NA
Signature (proxy for illiteracy) Educational attainment
Relationships to family members and witnesses Position in Household
Date (and hour) of death NA
![Page 15: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/15.jpg)
Combining data across levels of aggregation
Historians have rarely tried to combine data from sources of unequal levels of aggregation
Three approaches to combine data from the HSN and Censuses:
1. Aggregating individual data2. Multi-level or cross level analysis3. Disaggregating aggregate data
![Page 16: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/16.jpg)
Aggregating individual data
• Most straightforward way of combining two sources
• Details of the individual will be lost
• Aggregating HSN data for cross-sections at census years is no easy task
• Censuses are not perfect; statistical deviations found can either be caused by HSN or by census
![Page 17: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/17.jpg)
Multi-level analysis• No actual linkage of records; in multi-level
analysis the objective is to statistically explain a phenomenon in which higher levels of scale are included in the analysis
• Censuses provide background variables not available in HSN; whereas HSN contains individual detail not found in census tables
• In analyses at the individual level, ecological effects of higher levels may be taken in consideration
![Page 18: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/18.jpg)
Reconstruction of individual records from aggregate tables• Statistical Disclosure Control & synthetic
estimation methods– Prevent identification of individual entities from
aggregate data– Synthetic estimation methods can be used to
reconstruct synthetic individual records from detailed census tables
• Ecological inference– ‘…is the process of extracting clues about individual
behaviour from information reported at the group or aggregate level’
– Difficult technique, it remains a challenge to apply it to the Dutch censuses
Questions? Contact Peter Doorn ([email protected])
![Page 19: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/19.jpg)
Conclusion and directions for future research
This paper makes a plea for more interest by historians for the linkage of data from different levels of aggregation
The next step is to elaborate on the approaches described in this paper empirically
Data and techniques are available, we need a researcher who wants to take on the challenge
![Page 20: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/20.jpg)
Contact Information• dr. Peter Doorn
Director Data Archiving & Networked [email protected]
• drs. Luuk Schrevenproject coordinator Census [email protected]
Paper available in electronic form from website www.lifecoursesincontext.nl
![Page 21: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/21.jpg)
![Page 22: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/22.jpg)
![Page 23: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/23.jpg)
![Page 24: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/24.jpg)
![Page 25: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/25.jpg)
![Page 26: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/26.jpg)
![Page 27: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/27.jpg)
![Page 28: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/28.jpg)
Volksmenigte in de Bataafsche Republiek, 1795
0
100000
200000
300000
400000
500000
600000
700000
800000
900000
Provincie
Ziel
en
0
200
400
600
800
1000
1200
1400
1600
1800
Gro
ndve
rgad
erin
gen
Zielen
Grond-vergaderingen
![Page 29: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/29.jpg)
Population per municipality in 1795
Source: http://www.nidi.nl/
In 1795 Amsterdam is the biggest city with 217.024 “souls” Klein-Waspik is the smallest hamlet with 3 inhabitants; a total of 1807 municipalities are mentioned in the census.
![Page 30: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/30.jpg)
![Page 31: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/31.jpg)
![Page 32: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/32.jpg)
Boonstra’s NLKaart• Dr. Boonstra’s NLKaart;
– mid 1980’s onwards
– first Historical GIS?
– municipal boundariesbetween 1830 - 1990
– first SAS/Graph based,later MapInfo
![Page 33: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/33.jpg)
HGIN; a Historical Geographic Information
System for the Netherlands
• Project goals:– Converting and correcting Boonstra’s
NLKaart– digitizing maps with sub municipal
boundaries ‘wijken’ (neighbourhoods) and ‘buurten’ (blocks) (1920 – 1971)
– Setting up a gazetteer of historic places– Making the everything available on the web
![Page 34: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/34.jpg)
HGIN details: technical stuff
• Scalable Vector Graphics
• Geoserver as basic geographical data server (OpenGIS)
• User friendly interface in NIWI’s Content Management Software: i-Torsee: http://www.itor.org or http://www.nidi.nl for working preview of the GIS.
![Page 35: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/35.jpg)
HGIN details: results so far
• Testversion of mapping application is running at NIDI’s website (www.nidi.nl)
• 1960 and 1971 sub municipal maps available
• 1930, 1947 and 1956 maps are being digitized (outsourced)
• NLKaart converted to ArcGIS
• Work on gazetteer started
![Page 36: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/36.jpg)
HGIN details: religion 1971 (provincial)Percentage Roman Catholic by province, Census 1971
![Page 37: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/37.jpg)
HGIN details: religion 1971 (municipal)Percentage Roman Catholic by municipality, Census 1971
![Page 38: Peter Doorn & Luuk Schreven peter.doorn@dansdata.nl & luuk.schreven@niwi.knaw.nl](https://reader036.fdocument.pub/reader036/viewer/2022081604/56814bc6550346895db89c63/html5/thumbnails/38.jpg)
HGIN details: religion 1971 (submunicipal)
Percentage Roman Catholic by block / neighbourhood, Census 1971