1.Introduction
Role theory provides a productive framework to understand and explain foreign policy. The theory argues that foreign policies of states are directed by the role conceptions that are established by foreign policy-makers. Just like human beings in social life, states adopt certain roles and conduct their foreign relations accordingly. Hence, the initial step in analyzing foreign policy is to identify the national role conceptions (NRCs). Therefore, the majority of the studies that utilize the role theory framework focus on identifying the NRCs and their sources. A variety of methods can be used for such a purpose including interviews, public opinion polls, and content analysis. Some studies apply a combination of these different tools to increase the validity of their claims.
In my research I have developed an original approach through a combination of two research methods: content analysis and event data analysis. It is an eclectic approach combining the powerful sides of qualitative and quantitative approaches to research. In this article I give a step-by- step explanation of how I developed this approach and then show an empirical application: the Turkish Foreign Policy Roles and Events Dataset (TFPRED).[1] The TFPRED is a dataset including coded information on Turkey’s foreign policy words (NRCs) and deeds (role performance).
After explaining the eclectic methodological approach I analyze Turkish foreign policy (TFP) in the Justice and Development Party (JDP) (Adalet ve Kalkınma Partisi or AK Parti) era by utilizing the TFPRED. Applying role theory as a tool of analysis provided a comprehensive and structured framework to consider material as well as non-material variables that affect the foreign policy orientation(s) of the country. The theory is a useful tool to measure the impact of Turkey’s foreign policy makers’ vision on the actual foreign policy behavior of the country.
The temporal domain of this study starts from March 2003 and ends in August 2014. In this period JDP established three governments (the 59th, 60th and 61st). I compare the words and deeds of the three governments and build a generic methodological model that is applicable to other periods as well. With this methodological approach, I establish two main claims. First, there is a need to analyze both the discourse of the decision-makers’ (TFP words) and the actual practices of the country (TFP deeds). TFP decision-makers have certain role conceptions that they expect their country to perform. However, all role conceptions are not performed in the country’s foreign policy practices. Therefore I needed a two-phased data collection process: Data collection phase I – Content Analysis, and data collection phase II—Event Data. Second, Turkey’s relationship with one region to another varies, and analysis of the JDP decision-makers’ speeches indicated that some role conceptions have certain regional directions. Therefore, there was a need to categorize roles in terms of their regional direction. Building on this need, I proposed a new categorization for role conceptions: region-specific (Rg) roles and general/overarching (Ge) roles. Rg roles are divided into three sub-types. (1) Rtype1 roles are those that are directed to a single region. (2) RType2 roles are those that refer to multiple regions. (3) RType3 roles are those that refer to all regions. Finally, (4) Ge roles are those that have no specific regional direction.
The above-mentioned two claims are results of certain methodological choices. The following sections provide a structured explanation of how I made these choices. To this aim, I have divided this article into three main parts. First, I give a brief review of the literature on different applications of role theory framework and the use of event data in FPA. Second, I give a step-by-step application of these two methods, with examples of coded speeches, data sheets, news reports, and specific portions of the TFPRED dataset. Third, I give a brief summary of the empirical results of this methodological approach on TFP together with certain informative charts and graphics. In the conclusion, I make a critical assessment of the limitations of this methodological approach.
2. Role Theory Framework: Combining Content Analysis and Event Data
2.1. Role theory and the need to use event data
Holsti’s study on national role conceptions (NRCs) is a seminal work that established the roots of role theory[2]. The study provides a classification of states based on the NRCs of state leaders. Leaders, as Holsti argues, have cognitive perceptions about the roles that their state can play in the international arena. Their observations of the international environment are filtered through these perceptions, and their behavior is conducted accordingly. State foreign policies are conducted with an attempt to fulfill the role conceptions that the foreign-policy makers formulate. Based on a content analysis of leaders’ speeches from 1965 to 1967, Holsti identifies seventeen foreign policy role conceptions of the leaders in the era.
Since role conceptions are argued to determine state foreign policy activities, identifying such role conceptions in leaders’ speeches is proposed as a way to predict future foreign policy behavior of states. On this framework, Walker[3] adds that role conceptions are not just self-attributions of leaders but they are also based on the interaction of those leaders with the international environment. Roles are not just determined by leaders themselves but they are also affected by the events that happen in the international relations of those specific states. Reviewing role theoretical studies, Thies[4] similarly argues that although role conceptions can be analyzed through leaders’ speeches, they could also be extracted through a study of states’ foreign policy actions. Accordingly, I argue that a list of foreign policy role conceptions can be determined both with reference to the ‘discourse’ (leaders’ speeches) and the ‘practice’ (foreign policy actions) of states.
The existence of various role conceptions means that countries do not generally play single roles but in some cases they play multiple ones. In the same line of argumentation Holsti[5] also argues that multiple roles could be determined and played by states. Hence, the existence of multiple roles might lead to role conflict in some cases. For instance, Barnett [6] in his analysis of the Arab state-system finds out that there is a conflict between roles emphasizing state-sovereignty and the ones emphasizing pan-Arabism. Similarly, Tewes[7] identifies a role conflict between Germany’s stances on the issue of the EU enlargement and integration. Role conflict in such cases might either lead to a choice between conflicting roles or to a re-interpretation of specific role conceptions.[8]
Indeed, role theory has been fruitfully utilized as a practical tool in FPA as a rich literature has been produced since Holsti[9]. Granatstein[10] focused on Canadian role conception that motivates the country for initiating peacekeeping activities. Hedetoft provides a role theoretical analysis of the UK, Denmark and Germany. He emphasizes the relationship between the self-images of these states, their war mentalities, and their foreign policy behavior.[11] Breuning [12] studies the discourse and practice of UK, Germany and Netherlands with regard to the issue of foreign aid. Chafetz et al.[13] analyze Belarus and Ukraine’s foreign policy with role theory. Baehr [14] focuses on Netherland’s role conceptions. He argues that the country has assumed to be at the center of international law and its foreign policy is conducted accordingly. The literature is rich of such country studies.[15] It has also been applied to the Turkish case in a number of studies.[16]
Most of the studies that apply role theory have generally focused on identifying NRCs. Indeed, only a very limited number of the studies focus on how those role conceptions turn into foreign policy practice. Then, the argument is well-presented in the following lines[17]:
Most of the studies that apply role theory (…) have applied systematic methods (mostly content analysis) to analyze different leaders’ foreign policy speeches.(…) these studies do not systematically observe how much these role conceptions affect states’ foreign policy practices. Focusing on the sources of role conceptions and explaining them in detail only provides a partial application of role theory and does not fulfill the theory’s promise. (…) While developing a well-structured focus on the independent variable (role conceptions) of the theory, the existing literature has neglected a systematic focus on the dependent variable (foreign policy practices).
To put it boldly, the theory assumes that states’ foreign policy behavior is determined by the NRCs. These role conceptions are identified in the speeches of the decision-makers, through conducting interviews, analyzing official documents, or through the use of public opinion polls.[18] The literature is rich in this sense. In other words, the methods used to identify role conceptions and their sources are abundant. Then, an important question follows: “To what extent do these role conceptions turn into foreign policy practice?” The answer necessitates a tool to measure the parallelism between the conceptions and actual practices:
Together with an analysis of role conceptions, there is a need for a clearly explained, consistent, logical and systematic focus on the foreign policy practices of states. This focus bears a potential to bring significant contributions to the explanatory power of role theory and fill-in a significant gap in the literature.[19]
I accept that the analysis of state behavior starts with the impact of decision-makers’ visions, and identifying such visions is a great contribution by itself. Yet, in order to have a complete analysis of foreign policy, one needs to focus on what states actually do as well; that is, states’ actual practices. While NRCs can reveal reflections of policy-makers’ visions, we also need data on the actual practices and a tool to observe any parallelism between the two. I offer event data as a useful method to fill-in this gap in Role theory.
Although they do not use the role theory framework, a similar motivation has possibly been one of the major incentives for scholars who develop large-scale events data sets. To get a grasp of what states actually do or how states actually behave in their foreign relations. Event data supports scholars in identifying certain patterns of foreign policy through providing empirically grounded observations of the actual foreign policy behavior of states. “Event data are generated by examining thousands of newspaper reports on the day-to-day interactions of nation-states and assigning each reported interaction a numerical score or a categorical code”.[20] McClelland’s[21] World Event Interaction Survey (WEIS) and Azar’s[22] Conflict and Peace Data Bank (COPDAB) initiated the first two events data-sets. In both datasets the interactions between states in world politics are coded according to a specific coding scheme.
Being among the first event datasets, WEIS codes public events reported daily in the press. It identifies the flow of action and response between countries. The assumption is that “nations act consistently enough so that their past behavior is a source for the prediction of their present and future actions.”[23] Through analyzing the past actions of states, the WEIS dataset attempts to reveal patterns of interactions between countries. McClelland analyzes if these patterns are observable in international relations. An ‘Event’ is taken as the unit of analysis and it is described as “Each event/ interaction is a daily report of an international event.” Using the New York Times as its data source, the dataset codes five variables are coded: time, actor, target, arena, and event classification, and 22 event categories, which are themselves divided into multiple sub-categories, resulting in 63 categories in total. Each observed event is coded according to an issue area such as the Arab-Israeli Conflict, the 1967 war, the Cyprus issue, and the Vietnam conflict (along with 23 other issue areas). All countries in the world and some non-governmental actors are included in the dataset.[24]
COPDAB (Conflict and Peace Databank)[25] is a computer-assisted dataset covering the international and domestic events that happened between 135 countries from 1948 to 1978. In COPDAB, event statements are collected from public sources and transformed into ‘descriptive events’ (who said or did what to whom and when). Nation-states are taken as actors that act towards their domestic and world targets. Each event is aggregated by time periods, and the changes in the relations of states are interpreted statistically. Event statements are transformed into ‘descriptive events’, each one containing information on the date, actor, target, source (where/the public source that the event statement is gathered), issue type (military, economic, etc.), scale (Azar’s scale on relative degree of ‘Conflictiveness’ and ‘cooperativeness’ of the event), and the issue area. Similar to WEIS, COPDAB also categorizes events. Each country, public source, event type and the scale of each event has a specific code explained in the codebook. So an event statement like: “May 23, 1968 -- Egypt loans Syria $750,000 for Economic Development” will be coded as the following: “680520365165202LOAN 2004$750,000 ECONOMIC AID FOR DEVELOPMENT”.
WEIS and COPDAB are of significant importance since most subsequent event datasets have utilized the coding scheme developed by these early examples. For instance, CASCON (Computer-Aided System to Handle Information on Local Conflicts)[26] analyzes crisis events under a case/phase categorization. CREON, which was formed in 1969,[27] recorded events, defined as “any action resulting from a decision by political authorities of a state that has the power to commit the resources of the national government.” [28]
The SHERFACS[29] dataset builds data on international conflicts based on a crisis/phase scale. In SHERFACS, each event is coded under the six-phases of crisis. KEDS (Kansas Event Data System) utilizes the WEIS categorization and provides a machine-coding system for generating event data.[30] The KEDS System also collaborated with PANDA (Protocol for the Assessment of Nonviolent Direct Action) developed by Bond et al. at Harvard. [31] New verb categories have been developed in collaboration with the variety of event datasets utilizing the KEDS system. These categories have been built into two major event data systems - IDEA (Integrated Data for Events Analysis) and CAMEO (Conflict and Mediation Event Observations). CAMEO is based on the previous coding systems developed through the KEDS system. Another version of the most recent event data-coding scheme is IDEA developed by Bond et al.[32] Last but not the least, TFAED (the Turkish Foreign Affairs Event Dataset[33]) initiated a study on TFP between 1990 -2013.
Event data is based upon recognizing patterns of state behavior. Through a step-by-step process, researchers collect stories, code data, and find generalizable patterns through quantitative analysis in order to obtain predictive capacity on possible future behavior. Previously, datasets like WEIS and COPDAB relied on human analysts, basically collecting news reports and hand-coding each event in the dataset. However, later versions of event data such as KEDS and CAMEO utilize a software program called “Text Analysis by Augmenting Replacement Instructions (TABARI)” which is developed specifically for the purpose of event data generation. Recently, Schrodt and Yonamine, and Tüzüner and Biltekin gave a step-by-step explanation of developing event datasets.[34] Those steps are benefited from in the developing of the TFP Roles and Events Dataset (TFPRED) for the purposes of this study.[35]
2.2. Combining content analysis and event data to analyze TFP: A step-by-step explanation
The utility of role theory is that it provides a structured framework for foreign policy analysis. Indeed, in most studies NRCs are presented as the end products of foreign policy analysis. However, as noted above, I argue that NRCs should not be seen as the end-product but rather the starting point of foreign policy analysis. Once understood in that way, I believe role conceptions are a good way of summarizing and explaining the decision-makers’ vision in a compact conceptual manner. While defining each role conception that is identified, the researcher also summarizes the decision-makers’ foreign policy vision in a very compact manner.[36] In this section I will provide a detailed explanation on how to build a methodological tool to analyze both. In TFPRED, I coded 87 foreign policy speeches (approximately 238,000 words) of Turkish decision makers and approximately 36,000 news reports.[37] The following section provides a step-by-step explanation of the research procedures that I applied in TFPRED to combine content analysis and event data.[38]
2.2.1. Hand-coded content analysis coding scheme
RT Step 1 - Deciding on the method: The first step requires the researcher to decide on the proper method to identify NRCs. For various reasons including resource availability, time-management, and financial constraints, I decided to use content analysis of the foreign policy speeches of JDP decision-makers. I looked at the literature and the speeches of the leaders, found 22 role conceptions and coded data according to the coding scheme explained in the next steps below. [39]
Table 1 - TFPRED Role Conceptions
The coding scheme can only be prepared after determining the research question(s).[40] The coding scheme should (at least) explain the a) sampling method (selection of the raw material), b) coded information (What to code?), and c) the measurement strategy (How to code it?). Once the scheme is determined then one can generate content analysis data by choosing a sample of speeches. In TFPRED the sampling method refers to the rules that I applied in selecting the leaders’ speeches. For scientific reasons, one needs to have a clear, replicable and certain strategy in selecting the sample to be analyzed. The study (TFPRED) used a systematic sampling method[41]. That is to say, before including a speech in the sample, I confirmed that the speech was related to the subject matter of the study. So, a speech was only selected if its main topic was TFP. I also applied additional rules: a) the speech should be longer than 1000 words, b) the speech should be on general TFP not region or issue-specific, and c) the speech should include at least three role references. Next, I used purposive sample selection. The purposive sampling included speeches of Ministers of Foreign Affairs and Prime Ministers. After selecting 87 speeches with this strategy I started coding each speech separately.RT Step 2 – Preparing the content analysis coding scheme: Content analysis is generally done with a purpose, which is framed with a clear research question. The data in this study were collected with a purpose to answer the following: “What are the most frequently referred to foreign policy role conceptions in the selected speeches?” Relying on my role theory research framework, I divided this question into a list of five sub-questions: 1) How many different role conceptions are uttered and how do they differ from one government period to another?; 2) Do Leaders’ role preferences differ from one another or is there a consistency between them?; 3) Which roles are most/least preferred by each leader?; 4) Do role conceptions have regional-orientation or general-orientation?; and 5) Which regions/countries are the role conceptions directed at?
Here an important step is to determine which information to code. Especially hand-coding is a time-consuming and difficult task. In order to save time for future studies an important tip that should be noted here is that one should “code as much as much information as possible”. Data collection should first and foremost focus on collecting information relevant to the research questions at hand, but one should also keep in mind that some side questions might also be answered as one starts content analysis. In TFPRED, all speeches are coded separately in an MS Excel Worksheet, and each word, paragraph and context that refer to specific foreign policy role conceptions are coded separately. The next step is to decide on the measurement strategy.
In TFPRED, the role conceptions were coded with a specific system of enumeration. For the purposes of TFPRED I needed to measure three main subcategories: 1) frequency, 2) space, and 3) regional orientation. Frequency simply counts whether a foreign policy role occurs and how often it is referred to. I observed the percentage shares of each role as well. Space discovers how long the role is stated. For instance, in how many different contexts and paragraphs is the role elaborated? How many sentences or words refer to the role? Orientation is coded in order to detect the regional direction of the role references. Is the role directed towards a specific region (Rg) or does it have general orientation (Ge) without any regional direction?
RT Step 3: Coding Data on TFP Role Conceptions: Once the coding scheme is determined the next step is to prepare code sheets. The coding sheet should include space for all necessary information. Each speech will be coded to a separate code sheet. In TFPRED I had 22 roles. According to the coding scheme described above, a sample code-sheet will look something like the following:
The code sheet above has space for all information that I need. In TFPRED, a sample coded sheet looks like the following.
The codes in each column of the coding sheet refer to the following
Y/N : if the role is observed at least once in the speech, then I code “1”, if not “0”
F : How many times did I observe the role in the speech?
S-c : S(Space) –C(Context) in how many different contexts is the role referred to?
S-p : S(Space) –P(Paragraph) in how many paragraphs is the role referred to?
OR : The orientation (Rg or Ge) of the role
Rg (1/0): If any of the references to the role is directed at a region I code 1 if not 0
Rg# : How many times is the role directed at a region?
Ge (1/0): If any of the references does not have a regional direction I code 1 If not 0
Ge# : How many times is the role uttered with no Rg direction?
R/C : R (Region) – C (Country)
Rg1 : The Balkans and Eastern Europe
Rg2 : The Caucasus, Central Asıa, South Asia
Rg3 : MENA and Eastern Mediterranean
Rg4 : Sub-Saharan Africa
Rg5 : Euro-Atlantic
# : Number of references
C : Which country did the role refer to? If no C is found then code NA.
Each speech has to be coded in a separate code-sheet. Then the data is aggregated in an MS Excel Workbook.[42] The following table is taken directly from the dataset and shows how the aggregated version of the TFPRED content analysis data looks like.
RT Step 4- Interpreting data: Once the data is collected in the desired manner then one can start interpreting it. TFPRED separates and aggregates leader-by-leader and government-by government data from 2003 to 2013. I utilized MS Excel charts and graphs to compare and interpret the findings. Some examples are provided in the sections below. But before starting interpretation let us look at the necessary steps for the second data collection method: Event data.
2.2.2. Event data coding scheme
Event data coding techniques are abundant. However, the utility and accessibility of each technique is determined by the capabilities of the researcher. First, the software programs are not particularly user-friendly. Each coding software program works in the basic Terminal (MacOS) or Command Prompt (Microsoft) environment. Understanding and utilizing these coding programs takes time and requires basic knowledge of Linux/Unix/MsDos commands. In some instances, one might need to modify the software program codes in order to make proper use of them. So, a researcher needs to learn (at least) basic programming language in order to generate machine-coded event data. It is easy to reach these software programs from the websites of the above-mentioned datasets. However, the news sources where the raw (original/not coded) material is downloaded are not always found free of charge. So, the coding scheme greatly depends on at least two conditions: first, the financial resources of the researcher and second, access to basic computer programming capabilities.
ED Step 1: Deciding on the Relevant Software and Finding the Machine-Readable Source Text: There are a couple of ways of finding the source but the options are limited according to the institutional and financial capabilities of the researcher. Once the sources are found, then comes the limitation of the version of the software program that the analyst will be using in the coding process. For instance, the most recently developed program by event data analysts, PETRARCH (Python Engine for Text Resolution and Related Coding Hierarchy), reads events from .XML files. However, I have not been able to find a free newswire source that provides downloadable .XML files. Since Bilkent University provides access to LexisNexis Academics, a web source that provides downloadable files with the .HTML .TXT and .PDF file formats. I decided, following a similar method utilized by Tüzüner and Biltekin,[43] to utilize the latest version of the previous event data software TABARI (Textual Analysis by Augmented Replacement Instructions – Version 0.8.4b2), which is able to read .TXT files.
Once the original raw source files are acquired, then the researcher needs to use (at least) three sets of software programs to generate and interpret event data. First, filtering software is needed to convert the original TXT file into a TABARI-readable .TXT file. This software filters the unnecessary details in the newswire document, keeps the date and the ‘lead’ sentences of each news story and sorts them according to the release date. Then, the filtered files are processed through the TABARI software, which generates event data in the “Who, did What, to Whom, When” structure. Remember here that the resulting output.TXT file of the TABARI software will have hundreds of thousands of lines of events in the “Who (Target), did what (event), to whom (target), and when (date)” format, which makes thousands of pages in a single TXT file waiting to be analyzed and interpreted. This analysis and interpretation is done through utilizing the basic aggregation functions of some statistics software programs. There are a couple of ways of making these interpretations. In TFPRED, I utilized a combination of Rstudio to make data aggregations and MS Excel to generate informative figures and graphics.
ED Step 2: Keywords and Searching the LexisNexis Database: For the purposes of my study, I decided to utilize the free event data generating software programs of the Computational Event Data System (CEDS), TABARI (Textual Analysis by Augmented Replacement Instructions), provided by Parus Analytical Systems.[44] The programs work with two major online databases: LexisNexis (for AFP Newswire) and Factiva (for Reuters) as news sources. I decided to utilize the LexisNexis database.[45] Downloading the online sources requires the researcher to decide on the exact keywords, and source type. At this step I used a modified version of the method utilized by Tüzüner and Biltekin[46]. Tüzüner and Biltekin’s TFEAD Dataset searched for all types of events between a larger range of actors than the study that I was planning to conduct in this study. For the purposes of my study, I was specifically looking for the events conducted one-way by Turkish government actors, towards international actors. So, the keywords needed to be determined accordingly. The keywords that I used for my search were “LEAD (Turkey OR Turkish OR Turk OR ANKARA).”
After deciding on the keywords, I entered them into the advanced search option and tried to search for the whole temporal domain of the study, between November 18th, 2002 and August 29th 2014. I soon realized that for some unknown reason the LexisNexis database only shows the first 1000 of news reports if the search results are expected to come up with more than 3000 reports, so I had to divide each year into six months in order to make sure that the results came up with fewer than 3000 results. Then, another surprising result of the great wisdom of the LexisNexis Database engineers was that a researcher could only download 500 news reports in a single TXT file, therefore requiring multiple .TXT files. After finishing my LexisNexis journey, conducting approximately 23 separate “search-find” and 73 “download .TXT file” sessions, I downloaded 73 separate .TXT files each containing 500 news reports and saved them as raw material on my desktop. The search ended up with approximately 36,000 news reports from November 2002 to August 2014, making hundreds of thousands of pages saved in 73 separate TXT files. These files were formatted, divided into paragraphs, and made ready for TABARI by the filtering programs explained in the next section.
ED Step 3: Filtering and Reformatting Raw Material: The separate .TXT files downloaded from the LexisNexis website need to be merged, filtered, and sequentially sorted in order to create a single input file for TABARI. This was done with the use of three programs downloadable from the Parus Analytics website.[47] First, the "NewNexisFormat.pl” program runs through the news stories and re-writes their lead sentences into separate .TXT files. Then, the “nexisreverse.pl” program merges those separate files into a single TABARI readable .TXT file. Third, the “seqsort.pl” program does a date sort on the records if they are out of order.[48] After the use of these three programs, the NEWS reports that are downloaded from LexisNexis turn into a single TABARI readable text file containing the lead sentences of each report. The filtering process is very important to generate event data, otherwise, TABARI will not be able to read the news. The newly created and filtered single .TXT file becomes the input file for TABARI. The input file that I created for the TFPRED dataset, named “TFPRED160523.text,” contained approximately 150,000 lines of news reports (only lead sentences) from November, 2002 to August, 2014.
ED Step 4: Utilizing the TABARI Software and Coded events: For multiple reasons explained above I used the TABARI (Textual Analysis by Augmented Replacement Instructions – Version 0.8.4b2) program to code events.[49] The TABARI program folder can be downloaded from the Parus Analytics website. The folder contains most of the files that are necessary to run TABARI (the program file, options file, project file, and the actors and verbs dictionaries).
The TFPRED files were developed upon modifications of the GDELT dictionaries and options files provided by the Computational Event Data System Project. The GDELT dictionaries were also downloaded from the Parus Analytics website. In order to modify the dictionaries, first, a dry-run was made with the “Globalnet.131001” actors and verbs dictionaries. Then, events that are related to Turkey were analyzed and modifications made accordingly. In the dry-run, the filtered “TFPRED160523.text” file ended up with 14,204 events. Then the ministers of each government from 2002 to 2014, some city names from Turkey, and the Turkish version of the ministers’ and cites’ names (e.g. Gül, Erdoğan, Bağış, İstanbul, İzmir etc.) were added to the dictionaries. After modifications, a second run was made, ending up with 22,687 events. After further dictionary modifications another run was made, resulting again with the same number 22,687 events. The output file containing all 22,687 events was named “TFPRED160528”. This file contained events such as these:
021118 TUR ISR 090 Investigate
021118 IRQ USAGOV 195 Employ aerial weapons
021118 IRQ IGOUNO 128 Defy norms, law
…
140827 EUR IRN 036 Express intent to meet o
140828 TUR TURGOV 112 Accuse
140828 TUR TURGOV 112 Accuse
Once the events output file is created, then you can start working on event aggregation. Currently the “Events” package of the Rstudio (CranR) is the easiest and highly useful way to work on event data.[50] When the events are loaded into Rstudio the “events” package makes some additional tweaking of the data, removing duplicate events and sorting each event according to date.[51] The Rstudio-events package filtered data ended up with 20,187 events after the removal of mostly irrelevant (duplicate and sports related) events. However at this point the newly sorted and filtered data clarified that the TABARI software additionally codes phrases like “actor X said that it will do Y the next year” as if the promise has been made in the previous year. So, an event that took place in 14 March 2003 (Date: 030314) is coded as a promise that has been made the previous year (Date: 020314). Eighteen such instances were manually removed from the data set in order to fine-tune the events. As a result, the data ended up with 20,165 events between November 2002 and August 2016.
ED Step 5: A short note on Rstudio “Events” package: After working on the event data software (Text filtering, TABARI, PETRARCH) for approximately a year, the TFPRED data had finally become ready to aggregate and analyze. I had run the TABARI software a couple of times in order to reach the above-mentioned final document. Only some of the details are written here, however, I have prepared a long readme.txt file in the dataset and have put it in the TFPRED section of my website. Please note that, in my dissertation[52] I only provide a necessary summary of what I did to prepare Rstudio.[53] The document has details on how the R studio "events" package is used in filtering and aggregating the data to be utilized in the TFPRED data set.
Each step in building a data set requires a decision and each decision has certain consequences. In my work, I decided to utilize two programs - "R studio" and "MS Excel" and this is a result of many trials and failures. So the information in the document is based on my personal experience for the specific purpose of building the TFPRED data set. The document assumes that the reader has basic knowledge of R studio. If not, I still provide as much detail as possible but a best suggestion to those who would like to generate event data and aggregate with Rstudio might be a quote from Samuel Barclay Beckett: "Ever tried, ever failed, no matter, try again, fail again, fail better."
2.2.3. Matching the events with role conceptions
TFPRED observes the frequency and regional direction of 22 role conceptions and summarizes TFP practices via presenting data on the frequency (# of appearance), nature (conflict/cooperation continuum), and direction (regional orientation) of the events. After presenting TFP discourse and practice separately, I proposed a novel approach in observing the parallelism between the two. As shown in my dissertation; Appendix D 208 different events are coded in the dataset. In order to observe the parallelism, my study matched each of these events with relevant role conceptions. This Event-Role Matching Table is presented below:
With the matching table above one can observe the appearance of each role conception in the foreign policy events of the country. Next, let us see how the parallelism between role references and events are observed
2.3. A combination of role theory and event data: the Case of Turkey
This section gives short examples about the application of this methodology to the case of Turkey. Eighty-seven speeches are coded with the content analysis scheme explained above. The details of these speeches are as follows:
As indicated above I observed 22 role conceptions in TFP leaders’ speeches. All references to each of these 22 roles were coded together with the space, orientation, and regional direction of each reference. An excerpt from the overall data is already shown in the observed role references table above. From that table, I made a list of the most frequently and least frequently referred to role conceptions. The findings are summarized in the next table.
I calculated the percentages of role reference frequencies and made government based comparisons as well. From the data, I generated a number of exemplary figures in order to make certain interpretations.
Figure 1: Government-by-government role reference frequency
The charts above show the government-by-government role reference frequency percentages. The figures also indicate the percentage shares of each role conception. I used the percentages in order to compare the weight of each role conception in the speeches. I also used these percentage shares later in comparing role references (words) and role performance (deeds). Once data on TFP role conceptions were collected and interpreted, I focused on TFP deeds. For this purpose, I used TABARI to generate event data, aggregated the data and made my interpretations. With event data one can observe and compare at least three types of foreign policy behavior: 1. Frequency, 2. Scale, 3. Category.
First, one can look at the Frequency (intensity of relationship) of foreign policy events. Here, I count the number of observed events (Monthly, quarterly or yearly) and use the rising or declining number of events as indicators of TFP activism:
Figure 2: Number of events as indicators of TFP activism
From this figure one can make at least four main observations on TFP activism. First, TFP activity decreases before the general elections and makes a peak just after elections. Second, the number of observed foreign policy events increases when significant real life foreign policy crises (like revolutions, wars, political/military conflicts) emerge in the country’s surrounding regions. Third, there is a correlation (if not causality) between domestic political crisis and foreign policy activity. As the significance of domestic political crisis increase, the country’s foreign policy activity decreases. TFP activity falls below average when the country deals with domestic political turmoil like the E-Memorandum, Republican Meetings, constitutional disagreements, the terrorist bombing in Reyhanlı, the Gezi protests and the 17/25 December Investigations. Last but not the least, TFP activism follows a pattern of decline from one government to the other, between 2002 and 2014.
Second, one can apply a certain Scale to observe the nature of a country’s foreign policy behavior. Here, alternative scales exist but the most commonly used and widely accepted version in event data analysis is the Goldstein scale. Accordingly, I scaled events along the cooperation (+10) and conflict (-10) continuum. Then I aggregated these scores (quarterly or monthly) in order to observe the conflictual/cooperative nature of the country’s foreign relations. The figures emerged as follows:
Figure 3: The nature of TFP behavior
TFP event scores indicate three main observations about the nature of Turkey’s foreign relations. First, the early JDP governments (58th and 59th) followed a cooperative foreign policy towards the world. This period corresponds with the so-called “golden age” of Europeanization and the establishment of the “zero–problems” policy. The findings indicate that the initial activism in the JDP’s foreign policy was cooperative. Second, the TFP under the 60th government (July 2007- June 2011) turned into a mixture of conflict and cooperation. Third, the nature of the TFP turned towards conflict after 2010. As the foreign policy events towards the regions indicate, this turn towards conflict corresponds with the Arab Uprisings period in the MENA region. Overall, there is a significant pattern of decline in the average cooperation scores of the three JDP governments.
Third, one can look at the type of foreign relations by looking at their Category. Here, I categorized events along different event types, verbal cooperation/material cooperation, verbal conflict/material conflict. Then, I counted the number of cooperative and conflictual events and compared the type of foreign policy activism from one government to the other.
Figure 4: The type of foreign relations by looking at their Category
Event types also indicated similar findings with the event scales, with the JDP foreign policy turning towards conflict. As the figures indicate, the gap between the number of cooperative events and conflictual events closes. Cooperative events decrease while the number of conflictual events increases. The percentage share of cooperative events is 71% in the 58th and 59th governments, 67% in the 60th government, and 58% in the 61st government. Second, I also observed that the share of “verbal cooperation” is in decline. Third, there is a parallel significant pattern of increase in the share of material conflict: 5% in the 58th government, 8% in the 59th government, 12% in the 60th government, and 15% in the 61st government.[54]
After making these observations, I developed a novel approach to compare TFP Words (content analysis) and TFP deeds (event data). This comparison was made in four steps: (1) Match each event code with role conceptions; (2) observe the appearance of each role in TFP events; (3) calculate the percentage share of each role’s appearance; and (4) compare the percentage shares of each role in W/D. Then in comparing the words and deeds I applied the following scale:
Then, an excerpt from the TFPRED dataset is as follows:
From the table above, I excluded the roles that were non-observable with event-data and worked with the remaining roles.[55] The government based percentage shares of compared role conceptions in TFP words and TFP deeds are shown in the next figures:
Figure 5: The government based percentage shares of compared role conceptions in TFP words and TFP deeds
Once the percentage shares were compared, I generated the following summary table:
The table above illustrates that 11 roles are performed, five roles are underperformed and the parallelism of six roles could not be observed in the TFPRED.[56] From the words and deeds comparison I was able to make four overall observations. First, the TFPRED showed that all uttered roles are not performed in TFP practices. Second, there is a significant gap between W/D in peace-related roles. This indicates that TFP practices are not as peaceful as the decision-makers frame in their speeches. Certain peace related roles like Defender of Peace, Peacemaker, and Mediator are underperformed. Finally, both activism and cooperation are in decline in TFP. Turkey’s foreign policy activism is in decline and a turn towards conflict from one government to the other can be observed through this methodology.
3. Conclusion
This article offers a new methodological approach to analyze state foreign policies. With this approach a researcher can analyze foreign policy words, deeds, and parallelism between the two. Indeed, there is a wide and rich set of tools in the FPA literature that can be fruitfully utilized. However, most of the existing studies look at only one aspect of foreign policy while –somehow- disregarding the other. Foreign policy is the combination of both vision and action and the analysis requires a methodological tool that can identify the relationship between the foreign policy vision and its actual conduct. Thus, the claim here is that the relationship between the state foreign policy vision identified in the words or speeches of decision-makers and the actual foreign policy practices of states is not always direct. Vision does not always turn into action. In this sense, a holistic analysis of foreign policy requires a holistic methodological approach that can observe and identify the link between the two.
I have worked on this topic and other relevant ones for eight years and presented my research in several conferences. In most of those academic meetings I could only get mixed and limited feedback from the people who were not so busy to listen to me. Indeed, it was not always easy to get technical support to fine-tune the methodological approach that I was trying to develop. I could do the hard work by myself, hand-code and collect my own data, learn using new computer software by myself, yet for the methodological approach I needed support from experts. So it will be much easier if a beginner gets guidance and training from experts on the basics of data collection, use of software, and some tips and difficulties in the path. A qualified and encouraging supervision committee works perfectly in this case. Another way is to take part in research projects. During my graduate education, I took part in two different research projects where I assisted data collection. I strongly believe that experiencing teamwork and taking part in research projects is a perfect way to start data collection. At a certain point a young researcher wants to have more maneuvering capacity and to go beyond the limits that are pre-determined by another scholar’s research project. That point becomes the starting point of her/his own data collection journey.
Utilizing the above-mentioned eclectic methodology, I collected and categorized data on TFP words and deeds between November 2002 and August 2014 in the TFPRED. I categorized and aggregated data in order to make leader-specific, region-specific and government-specific observations on TFP. The TFPRED makes the following contributions: (1) it builds up a novel methodological model to combine observed role conceptions (words) with observed event data (deeds); (2) it builds a large open-source dataset on observed role conceptions and foreign policy events; (3) it provides a structured, comprehensive, and falsifiable explanation of TFP in the JDP period based on observable and replicable data; (4) it establishes a new role conceptualization of performed, underperformed and over-performed role conceptions; and (5) it proposes a new role typology to cover the regional direction of role conceptions. However, while these present positive steps forward, the comparison summary table shown earlier also revealed that the performance of some roles could not be observed, indicating the need to acknowledge certain methodological limitations.
Some methodological limitations stem from the use of event data analysis. As I have summarized above there are some studies that use event data in FPA. Yet, most of these studies utilize event data for conflict analysis. These studies analyze domestic events, but not foreign policy events. Because of this, some of the event verb codes are only related to domestic conflict. The coding software TABARI or PETRARCH are continuously developed with an aim to observe domestic or international conflicts around the world. The coded data includes data on actors other than states and issue areas other than foreign policy. The current state of the literature indicates that event data analysis will continue to develop in the ‘conflict analysis’ path and will increasingly part from the FPA field. While building the TFPRED, I filtered relevant actors and excluded irrelevant verbs from the data. The filtering, excluding, and aggregating relevant data, which is the approach that I apply in this study, will still be applicable in the future. Yet, a better approach at the issue would be to further update event data software, and increase its capacity to observe a larger set of foreign policy events with more foreign policy related verb codes. To achieve this aim, the theoretical and methodological approach of this study can be a good starting point.
Indeed, the study of foreign policy will continue to remain incomplete without having a scale to measure performance. The procedures that I apply in building the TFPRED dataset and in making the words-deeds comparison in this study aim at developing such a scale. Yet, although it has a high potential, I should admit that the scale needs further development. Perhaps, acquiring and applying more advanced statistical skills could increase the validity of the comparison.
Another and equally important point of assessment is about content analysis. I preferred to use hand-coding in content analysis in my research. There are quite advanced coding software programs that a researcher can use to analyze foreign policy speeches. Yet, I am not sure if I could rely on the capacity of these programs in analyzing Turkish speeches. Role conceptions are generally hidden in the context and the meaning attributed to words. For this reason, it is also necessary to fine-tune the definitions in the process of reading the speeches. Sometimes I even needed to add new role conceptions to the list while reading different leaders’ speeches just because of the meanings hidden in the context. One of the most important steps here is to create exclusive and exhaustive definitions of each role conception. This step will most probably take longer than expected. Having public opinion polls, making interviews to check the definitions with other experts, looking at the existing definitions in the literature and checking the inter-coder reliability by asking other people to code speeches with the existing definitions might be useful measures to have better definitions.[57]
After coming up with definitions of role conceptions that work for the research, an important next step is to observe the parallelism. Yet, here the researcher faces another significant problem. Some role conceptions are basically non-observable. In my study, I could not observe the performance of six role conceptions because I could not measure them with event data. I think that one can develop alternative ways to observe these role conceptions. For instance, the performance of ‘rising power’ role conception requires having a detailed look at economic, and military figures indicating the increasing power of Turkey. A ‘model country’ role conception necessitates fieldwork in the regions towards which the role is directed at. In addition, the performance of ‘energy transporting country (ETC)’ role conception requires taking a look at Turkey’s energy agreements with the countries of the regions. All these methods are currently beyond the scope of this study. Yet, these are fruitful directions for future research.
These limitations also determine the applicability of the methodological approach, which is, of course, always open for further development. The approach contains both advantages and disadvantages in its application. One of the important advantages is that the researcher can collect data. Data-collection is a burdensome yet very fruitful process. In addition to reviewing the existing literature a researcher needs to acquire certain skills to collect and code data. Learning the basics of data collection, obtaining the raw material (speeches or news reports) to analyze/process, taking necessary scientific measures, coding data, checking data validity, and convincing the scientific community/audience are difficult and time/energy consuming processes. Indeed, as indicated previously this is a journey where the researcher “tries again, fails again, and fails better.” However, the process usually brings very concrete, observable and applicable results. For instance, my research produced the TFPRED. Collecting data and combining words and deeds of foreign policy came up with a potentially very productive research agenda. In its current state, beside its contributions to the academic literature, the study stands as a ‘research questions generator.’
As a research questions generator, the TFPRED has shown the validity of my two main claims: 1) There are region-specific differences in Turkey’s NRCs, and 2) All role conceptions (words) do not turn into practice (deeds) in foreign policy. The TFPRED gives descriptive/statistical data on foreign policy vision, on the foreign policy practices, on the roles attributed to the country, and on the foreign relations of the country. It also shows both performed and non-performed role conceptions. With each performed and non-performed role conception one can come up with new research questions: (1) Why are some role conceptions performed whereas others are not? (2) Why do leaders utter role conceptions that are not performed? (3) If non-performed role conceptions are not to be taken as determinants of future foreign policy actions then why do we find them? On the other hand, one can also come up with country-specific or empirical research questions. For Turkey, the country is generally referred to as a middle power, or regional great power. Such status potentially affects the country’s role performance in the five neighboring regions (Balkans, MENA, Southern Caucasus/Central Asia, Sub-Saharan Africa, and Euro-Atlantic). In addition to status, role prescriptions are those that are ascribed to the ego (the agent- in IR the state) by the ‘alter’ (other actors within the society- in IR the countries/allies and/or other international actors). For instance, Turkey has been prescribed a bridge role between the democratic/Christian West and authoritarian/Muslim Middle East. For a long time, Turkish leaders have been eager to perform such a role. But the understanding of bridge has been slightly changed in the JDP period, at least in rhetoric. For instance, JDP decision-makers, especially in their first term of government referred to a mediator role in establishing an alliance between civilizations. However, it is hardly possible to claim that together with Spain the two countries were able to materialize such a role, except for routine meetings between the heads of the two states. Was this role beyond Turkey’s capabilities? Or was it due to Davutoğlu’s vision of becoming a central country, which clashed with the bridge role prescribed by Turkey’s Western allies? All these questions require separate in-depth research that builds on the data collected with the eclectic methodological approach.
It is always possible to update and upgrade this eclectic approach. One can increase the capabilities of this eclectic approach and/or the validity of the collected data by merging it with new research techniques. Just like content analysis and event-data analysis it is possible to bring in new research methods and combine it with the approach offered here. For instance, one can use semi-structured interviews to better understand the foreign policy vision of the decision-makers. In addition, other FPA approaches and tools like Leadership Trait Analysis (LTA) and Operational Code Analysis (Op-Code) can also be integrated with this approach to better understand how the decision makers’ observe their foreign policy environment and what kind of foreign policy they envision for their states. The quality of the collected data and the validity of the claims can be increased with the application of new techniques to this approach.
This study started five years ago, drawing on Rosenau’s call to develop a ‘theory’ of foreign policy.[58] After reading this call, I aimed at building a data-based and generalizable explanation of TFP and developing a sui generis framework to observe the parallelism in foreign policy words and deeds. This scheme is currently in a state of development and there is long way to go in this quest. After five years of work, I still agree with Hudson[59]: “It is a wonderful time to become engaged in FPA, a time of new horizons.”
