Where does Linux store its logs?

Linux stores logs in the /var/log directory, though not all applications need to abide by this soft rule.

Linux stores logs in the C:/sys32/logs directory, though not all applications need to abide by this soft rule.

Linux stores logs in the /usr/bin/log directory, though not all applications need to abide by this soft rule.

Linux stores logs in the /home/<user_name> directory, though not all applications need to abide by this soft rule.

What are the differences among encoding, encrypting, and hashing?

Encoding is meant to simply change the format of data to make it easily cross network and systems in order to make data widely available. Encrypting is a reversible function meant to keep data confidential. Hashing is a one way function that is used to verify data integrity.

Encoding is meant to simply change the format of data to make it easily cross network and systems in order to verify data integrity while making it difficult to see what is being sent. Encrypting is a reversible function meant to keep data confidential. Hashing is a reversible function that is used to verify data integrity.

Encoding is a reversible function meant to keep data confidential. Encrypting is a one way function that is used to verify data integrity. Hashing is meant to simply change the format of data to make it easily cross network and systems in order to make data widely available.

Encoding is a one way function that is used to verify data integrity. Encrypting is a reversible function meant to keep data confidential. Hashing is meant to simply change the format of data to make it easily cross network and systems in order to make data widely available.

What is Active Directory?

Active Directory is a service that stores information about users, computers, groups, and other objects in a network. It is a centralized way to manage and authenticate users and resources in a Windows domain.

Active Directory is a service that stores log information about users, computers, groups, and other objects in a network. It is a centralized way to manage and view events in a Windows domain.

Active Directory is a server in a domain that stores user account information, manages authentication and authorization, and enforces security policies.

Active Directory is a centralized way to monitor system and application events, troubleshoot problems, and diagnose issues in a Windows environment.

You are running a pen test on a company, what are the different phases you would undergo to perform penetration assessments?

Reconnaissance, Scanning, Vulnerability Analysis, Exploitation, Reporting

Scanning, Vulnerability Analysis, Exploitation, Reporting

Scanning, Reconnaissance, Vulnerability Analysis, Exploitation

Vulnerability Analysis, Scanning, Reconnaissance, Exploitation, Reporting

W hat are the phases of the digital forensics investigation process.

Identification, preservation, analysis, documentation, and presentation.

Preservation, analysis, i dentification, documentation, and presentation.

Identification, preservation, analysis, and documentation.

Identification, preservation, analysis, and presentation.

What is the difference between vulnerability, threat, and risk.

Vulnerability is a weak point in our system. Threat is a malicious actor that may exploit a vulnerability. Risk is the likelihood that a threat will actually take advantage of a vulnerability.

Vulnerability is an exploit in our system. Threat is a malicious actor. Risk is rating of how terrible our vulnerability is.

Vulnerability is identified by a CVE ID. Threat is a malicious actor that may exploit a vulnerability. Risk is the CVSS score.

Vulnerability is a weak point in our system. Threat is a malicious actor that may exploit a vulnerability. Risk is rating of how terrible our vulnerability is.

What are the core functions of the NIST cyber security framework?

Identify, protect, detect, respond, and recover.

Preparation, detection, analysis.

Identification, preservation, analysis, documentation, and presentation.

Detection, repeatability, documentation, respond.

Bash, or Bourne Again Shell, is a computer language interpreter and language. It makes command line editing easier and more convenient to use.

Bash, or Bourne Again Shell, is a scripting language used to automate tasks.

Bash, or Bourne Again Shell, is text editor for the command line that is interactive.

Bash, or Bourne Again Shell, is a simple command line terminal software without many bells or whistles.

What are the three states of data, and what types of encryption are used for each?

Data at rest, typically uses symmetric encryption. Data in transit, uses IPSec or VPNs. Data in use, which does not use encryption.

Data at rest, typically uses asymmetric encryption since its quicker. Data in transit, uses symmetric encryption. Data in use, uses elliptic curve cryptography.

Data at rest, typically uses symmetric encryption. Data in transit, uses IPSec or VPNs. Data in use, uses asymmetric encryption through Certificate Authorities.

Data at rest, typically uses symmetric encryption. Data in transit, uses no encryption since we are accessing it. Data in use, uses VPNs to secure data while we need it.

What is Group Policy?

Group Policy, is a feature of Windows that allows administrators to manage and configure settings for users and computers in an Active Directory environment.

Group Policy, is an IDS feature in Windows that allows administrators to detect when an event occurs.

Group Policy, is a feature of Windows that allows administrators to manage and configure settings for only users across the entire domain.

Group Policy, is a centralized way to enforce security policies, software deployment, network settings, and other configurations for computers across the entire domain.

Interview Questions Review

What is a rubric? A tool comprising a set of criteria (with possible levels of performance quality on the criteria) developed to assess learners’ work, from written to oral to visual. It is used tomeasureperformance,suchastheprocess of doing something (e.g.,playing a musical instrument, making a speech) or products of the learners’ work (e.g., concept map, laboratory report, bookshelf) (Brookhart, 2013). BENEFITS OF USING RUBRICS Support authentic assessment Reflects how well learners are able to apply knowledge inthe real-world context. Communicate expectations Gives learners an idea of what is expected of them. It is especially useful when the rubrics are communicated to the learners before they are assessed. Improve performance Explicit criteria and performance level descriptions allow learners to understand the desired performance. Learners are able to assess themselves by referring to the specific criteria and performance-level descriptions. Provide informative feedback Instructors are able to provide constructive feedback to learners on their weaknesses and strengths. Promote thinking andlearning 4 Provide informative feedback Instructors are able to provide constructive feedback to learners on their weaknesses and strengths. Learners are able to review and revise their work,thus reflecting on their learning experiences. Ensure fairness Learner performance assessed fairly given its objectivity. It helps avoid disputes between learners and instructors about the scores/grades achieved. TYPES OF RUBRIC ANALYTIC It consists of individual criterion with corresponding descriptor of performance. HOLISTIC It consists of performance descriptors that are placed together to refeclet to overalll performance. ANATOMY OF ANANALYTIC RUBRIC Rating scales with corresponding scores or weights The row represents the criteria for the desired performance, while the column represents the evaluation score. Under the rating scale (corresponding weights orscorescanbeassigned),theperformance descriptors are explicitly stated ANATOMYOF AHOLISTICRUBRIC Descriptions: It comprises the rating scale (corresponding weights or scores can be assigned) in the row while the combined desired performance descriptors are placed in the column. Description of the task The purpose of the assignment is to assess learner’s cognitive and analytic skills in applying knowledge gained and constructed throughout the course Diffusion of Innovation,bywatching the Surrogates movieand writing ananalytical review of the movie in the context of innovation diffusion.Iwant to provide learners with informative feedback on their cognitive and analytic skills such as the following: applying the concepts of innovation diffusion,making judgmentson the scenes related to innovation diffusion identified from the movie,selecting and critiquing theories of innovation diffusion and making connections between the theories,aswell asarguingand proposing necessary solutions to the problemss hown in the movie. ESTABLISHING ALTERNATIVEASSESSMENTINHIGHEREDUCATION VALIDITYAND RELIABILITYOF RUBRICS. Validity Measuring what is supossedto be measured. Reability Yielding consists results. Instruments that are used in the alternative assessment must be aligned to the learning outcomes and measure well what it intends to measure (valid) and produce consistent scores (reliable). The valid instrument will manifest the true ability (latent trait) of learners and permit appropriate inferences to be made about a specific group of people for specific purposes. TYPES OF VALIDITY FACE VALIDITY Simple form of validity thatapplies a superficial and subjective assessment whether the instrument measures what it is supposed to measure. CONTENT VALIDITY Refers to the extent to which the items on a measure assess the same content or how wellthe content material was sampled inthe measure. CONSTRUCT VALIDITY Refers to the extent to which the test may be said to measure a theoretical construct or trait. CONCURRENT VALIDITY Refers to the extent to which scores onanewmeasure are related to scores from a criterion measure administered at the same time. PREDICTIVE VALIDITY Refers to the uses of the scores from the new measure to predict performance on a criterion measure administered ata later time. STEPS TO CONSIDER WHEN ESTABLISHING CONTENT VALIDITY Calculate the level of expert agreeement for the content validity, get expert to verfy. Interview the expert ,make meta contentdata análisis from literatura. STEPS TO CONSIDER WHEN ESTABLISHING CONSTRUCCT VALIDITY Administer the instrument for alll learners, revise any item necccesay, run an apropriates statistical analiysis, administerthe instrument to learners as a pilot test . CONSTRUCTMAP Morepreciseconceptthan construct. Ranges from one extreme to another(fromhightolow,small tolarge,positivetonegative,or strongtoweak). Identifiesthepositionofthe respondentsinthisrange. Representativenessofsampling (questions and ability of respondents). EXAMPLEO FACONSTRUCTMAP:AFFECTIVE LEVELOF AFFECTIVE VARIABLES EXAMPLESOFITEMSIN MEASURINGTEAM WORKING SKILLS 5. Characterisation Learnersvolunteerstodothe groupworks. 4. Organisation Learners are willing to help others,althoughitisnottheir scopeoftask. 3. Valuing Learners respect other team members’opinionwhendoing thediscussion. 2. Responding Learnergivescooperationwhen neededingroupworks. 1. Receiving Learneracceptsthediversityof races and nationalities among groupmembers. EXAMPLEOFACONSTRUCTMAP:PSYCHOMOTOR LEVELOF PSYCHOMOTOR VARIABLES EXAMPLESOFITEMSIN MEASURING DIGITAL SKILLS 7.Origination Learnerscanmodifytheirowndevicesto performbetter. 6.Adaptation Learnerscansolveandtroubleshootthe problemwhileusingthecomputer. 5.ComplexOvertResponse Learnerscanusethecomputercompetently. 4.Mechanism Learners can use the computer independently,butstillmakeminorerrors. 3.GuidedResponses Learnerscanusethecomputer,butstill needguidance. 2.Set Learnersarereadytousethecomputer. 1.Perception Learnerscanobservehowtousecomputer. EXAMPLEOFACONSTRUCTMAP:COGNITIVE LEVELOF COGNITIV E VARIABLES EXAMPLESOFITEMS IN MEASURING THINKINGSKILLS 6. Creating Learners are able to suggest anewmodelorframeworkof learningdigitalcommunity. 5. Evaluating Learners are able to judge the impactofthescenariotowards educationperspective. 4. Analysing Learnerscandifferentiate the factsusingafew theories. 3. Applying Learnerscansolveproblems usingthefactsgiven. 2. Understanding Learnersareabletoexplainthe factsusingtheirownwords. 1. Remembering Learnersonlymemorisethe. Direction of Increasing “X” Learners Learners with high “X” Learners with mid range “X” Learners with low “X” Responses to Item Item response indicate highest level of X Item response indicate higher level of X Item response indicate lower level of X The construct map shows the lower ability students are in line with the lower level of items. This shows that when educators plan to develop an instrument, it Item response indicate lowest level of X Direction of Decreasing “X” is crucial to create an item difficulty thatrepresents learners’ ability. Learners’ ability Learners who engage in level characterisation Learners who engage in level organisation Learners who engage in level valuing Learners who engage in level responding Learners who engage in level receiving Direction of Decreasing“X” MEASURINGCONSTRUCTVALIDITY Unlike content validity, this construct validity can be analysed using statistical analysis. Use Exploratory FactorAnalysis [EFA], Confirmatory FactorAnalysis [CFA] or Unidimensionality to confirm all items are measuring the right construct and the raw variance explained for the latent variables is sufficient. Gap initem map also can show accuracy in construct validity. RELIABILITY The degree to which test scores are consistent over repeated administrations of the same/ equivalent test and therefore considered dependable and repeatable for an individual learner.A test thatproduces highly consistent and stable results (i.e. relative free from random error) is said to be highly reliable. TYPESOFRELIABILITY Test-retest demonstrates the stability of a measure over time 01 Internal consistency most of the items within a rating scale of a concept show consistency of scoring. Inter-rater the extent to which two or more independent raters are consistent in observing, recording and scoring data (should be 70% or higher agreement) 04 Intra-rater relies on one rater to rate an object or event twice (70% or higher of agreement) FACTORSAFFECTING VALIDITYANDHOWTO INCREASEVALIDITY? FACTORS AFFECTING VALIDITY HOWTO INCREASE VALIDITY? 1. Inaccuracy of items in measuringtheoutcomes 1. Vetting session to get reviewsfromtheexpert. 2. Pooritemsdevelopment 2. Followtheformatandtips indevelopinggooditems. 3. Unclearinstructions 3. Do pilot testing to measuretheusabilityof thetest. 4. Interveningevents 4. Controltheinternalthreats validityfactors. 5. Itemsdifficultyisnot suitableforthelearners 5. Create a construct map toensurethereisanitem thatrepresentslearners ability. FACTORS AFFECTING RELIABILIT Y HOWTOINCREASERELIABILITY? 1. TestLength 1. Thetestlengthshouldbeappropriate withtestdifficulty. 2. Test retest interval 2. Suggesteddurationisbetween3 weeksto2months. 3. Variability of scores 3. Doconstructmaptoensuretheitems aresuitablewithlearners’ability. 4. Guessing 4. Penalisetheguessinganswers.You alsocandetecteitherthelearnersare guessing or not using the statistical analysis named guessing analysis andpersonfitanalysis. 5. Inconsistency score from different raters 5. Appointtheratertomarkcertain questionsforalllearners(Thisalways happen when you have more than onesectionandhavemorethanone lecturer). CONCLUSION Coming back to the issue of validity and reliability in assessment, there is a need for educators to put an effort to ensurethattheitemsintheformofquestionsorinstructions arenotonlyclearbutalsoabletomeasurewhatitisintended tomeasurebasedontherelatedlearningoutcomes. Establishingvalidityandreliabilityofinstrumentscan provide educators with some indications of the quality of the measuring tools being used. Valid and reliable instruments enabletheeducatorstocontinuouslyusethemeasuringtools withoutreservation. Reliablenot valid Precisenot Accurate Reliableand valid Preciseand Accurate NotReliable butvalid NotPrecisebut Accurate NotReliable butNotvalid NotPrecisebut NotAccurate 94

We found you were making a quiz on the subject of "What is a rubric? A tool comprising a set of criteria (with possible levels of performance quality on the criteria) developed to assess learners’ work, from written to oral to visual. It is used tomeasureperformance,suchastheprocess of doing something (e.g.,playing a musical instrument, making a speech) or products of the learners’ work (e.g., concept map, laboratory report, bookshelf) (Brookhart, 2013). BENEFITS OF USING RUBRICS Support authentic assessment Reflects how well learners are able to apply knowledge inthe real-world context. Communicate expectations Gives learners an idea of what is expected of them. It is especially useful when the rubrics are communicated to the learners before they are assessed. Improve performance Explicit criteria and performance level descriptions allow learners to understand the desired performance. Learners are able to assess themselves by referring to the specific criteria and performance-level descriptions. Provide informative feedback Instructors are able to provide constructive feedback to learners on their weaknesses and strengths. Promote thinking andlearning 4 Provide informative feedback Instructors are able to provide constructive feedback to learners on their weaknesses and strengths. Learners are able to review and revise their work,thus reflecting on their learning experiences. Ensure fairness Learner performance assessed fairly given its objectivity. It helps avoid disputes between learners and instructors about the scores/grades achieved. TYPES OF RUBRIC ANALYTIC It consists of individual criterion with corresponding descriptor of performance. HOLISTIC It consists of performance descriptors that are placed together to refeclet to overalll performance. ANATOMY OF ANANALYTIC RUBRIC Rating scales with corresponding scores or weights The row represents the criteria for the desired performance, while the column represents the evaluation score. Under the rating scale (corresponding weights orscorescanbeassigned),theperformance descriptors are explicitly stated ANATOMYOF AHOLISTICRUBRIC Descriptions: It comprises the rating scale (corresponding weights or scores can be assigned) in the row while the combined desired performance descriptors are placed in the column. Description of the task The purpose of the assignment is to assess learner’s cognitive and analytic skills in applying knowledge gained and constructed throughout the course Diffusion of Innovation,bywatching the Surrogates movieand writing ananalytical review of the movie in the context of innovation diffusion.Iwant to provide learners with informative feedback on their cognitive and analytic skills such as the following: applying the concepts of innovation diffusion,making judgmentson the scenes related to innovation diffusion identified from the movie,selecting and critiquing theories of innovation diffusion and making connections between the theories,aswell asarguingand proposing necessary solutions to the problemss hown in the movie. ESTABLISHING ALTERNATIVEASSESSMENTINHIGHEREDUCATION VALIDITYAND RELIABILITYOF RUBRICS. Validity Measuring what is supossedto be measured. Reability Yielding consists results. Instruments that are used in the alternative assessment must be aligned to the learning outcomes and measure well what it intends to measure (valid) and produce consistent scores (reliable). The valid instrument will manifest the true ability (latent trait) of learners and permit appropriate inferences to be made about a specific group of people for specific purposes. TYPES OF VALIDITY FACE VALIDITY Simple form of validity thatapplies a superficial and subjective assessment whether the instrument measures what it is supposed to measure. CONTENT VALIDITY Refers to the extent to which the items on a measure assess the same content or how wellthe content material was sampled inthe measure. CONSTRUCT VALIDITY Refers to the extent to which the test may be said to measure a theoretical construct or trait. CONCURRENT VALIDITY Refers to the extent to which scores onanewmeasure are related to scores from a criterion measure administered at the same time. PREDICTIVE VALIDITY Refers to the uses of the scores from the new measure to predict performance on a criterion measure administered ata later time. STEPS TO CONSIDER WHEN ESTABLISHING CONTENT VALIDITY Calculate the level of expert agreeement for the content validity, get expert to verfy. Interview the expert ,make meta contentdata análisis from literatura. STEPS TO CONSIDER WHEN ESTABLISHING CONSTRUCCT VALIDITY Administer the instrument for alll learners, revise any item necccesay, run an apropriates statistical analiysis, administerthe instrument to learners as a pilot test . CONSTRUCTMAP Morepreciseconceptthan construct. Ranges from one extreme to another(fromhightolow,small tolarge,positivetonegative,or strongtoweak). Identifiesthepositionofthe respondentsinthisrange. Representativenessofsampling (questions and ability of respondents). EXAMPLEO FACONSTRUCTMAP:AFFECTIVE LEVELOF AFFECTIVE VARIABLES EXAMPLESOFITEMSIN MEASURINGTEAM WORKING SKILLS 5. Characterisation Learnersvolunteerstodothe groupworks. 4. Organisation Learners are willing to help others,althoughitisnottheir scopeoftask. 3. Valuing Learners respect other team members’opinionwhendoing thediscussion. 2. Responding Learnergivescooperationwhen neededingroupworks. 1. Receiving Learneracceptsthediversityof races and nationalities among groupmembers. EXAMPLEOFACONSTRUCTMAP:PSYCHOMOTOR LEVELOF PSYCHOMOTOR VARIABLES EXAMPLESOFITEMSIN MEASURING DIGITAL SKILLS 7.Origination Learnerscanmodifytheirowndevicesto performbetter. 6.Adaptation Learnerscansolveandtroubleshootthe problemwhileusingthecomputer. 5.ComplexOvertResponse Learnerscanusethecomputercompetently. 4.Mechanism Learners can use the computer independently,butstillmakeminorerrors. 3.GuidedResponses Learnerscanusethecomputer,butstill needguidance. 2.Set Learnersarereadytousethecomputer. 1.Perception Learnerscanobservehowtousecomputer. EXAMPLEOFACONSTRUCTMAP:COGNITIVE LEVELOF COGNITIV E VARIABLES EXAMPLESOFITEMS IN MEASURING THINKINGSKILLS 6. Creating Learners are able to suggest anewmodelorframeworkof learningdigitalcommunity. 5. Evaluating Learners are able to judge the impactofthescenariotowards educationperspective. 4. Analysing Learnerscandifferentiate the factsusingafew theories. 3. Applying Learnerscansolveproblems usingthefactsgiven. 2. Understanding Learnersareabletoexplainthe factsusingtheirownwords. 1. Remembering Learnersonlymemorisethe. Direction of Increasing “X” Learners Learners with high “X” Learners with mid range “X” Learners with low “X” Responses to Item Item response indicate highest level of X Item response indicate higher level of X Item response indicate lower level of X The construct map shows the lower ability students are in line with the lower level of items. This shows that when educators plan to develop an instrument, it Item response indicate lowest level of X Direction of Decreasing “X” is crucial to create an item difficulty thatrepresents learners’ ability. Learners’ ability Learners who engage in level characterisation Learners who engage in level organisation Learners who engage in level valuing Learners who engage in level responding Learners who engage in level receiving Direction of Decreasing“X” MEASURINGCONSTRUCTVALIDITY Unlike content validity, this construct validity can be analysed using statistical analysis. Use Exploratory FactorAnalysis [EFA], Confirmatory FactorAnalysis [CFA] or Unidimensionality to confirm all items are measuring the right construct and the raw variance explained for the latent variables is sufficient. Gap initem map also can show accuracy in construct validity. RELIABILITY The degree to which test scores are consistent over repeated administrations of the same/ equivalent test and therefore considered dependable and repeatable for an individual learner.A test thatproduces highly consistent and stable results (i.e. relative free from random error) is said to be highly reliable. TYPESOFRELIABILITY Test-retest demonstrates the stability of a measure over time 01 Internal consistency most of the items within a rating scale of a concept show consistency of scoring. Inter-rater the extent to which two or more independent raters are consistent in observing, recording and scoring data (should be 70% or higher agreement) 04 Intra-rater relies on one rater to rate an object or event twice (70% or higher of agreement) FACTORSAFFECTING VALIDITYANDHOWTO INCREASEVALIDITY? FACTORS AFFECTING VALIDITY HOWTO INCREASE VALIDITY? 1. Inaccuracy of items in measuringtheoutcomes 1. Vetting session to get reviewsfromtheexpert. 2. Pooritemsdevelopment 2. Followtheformatandtips indevelopinggooditems. 3. Unclearinstructions 3. Do pilot testing to measuretheusabilityof thetest. 4. Interveningevents 4. Controltheinternalthreats validityfactors. 5. Itemsdifficultyisnot suitableforthelearners 5. Create a construct map toensurethereisanitem thatrepresentslearners ability. FACTORS AFFECTING RELIABILIT Y HOWTOINCREASERELIABILITY? 1. TestLength 1. Thetestlengthshouldbeappropriate withtestdifficulty. 2. Test retest interval 2. Suggesteddurationisbetween3 weeksto2months. 3. Variability of scores 3. Doconstructmaptoensuretheitems aresuitablewithlearners’ability. 4. Guessing 4. Penalisetheguessinganswers.You alsocandetecteitherthelearnersare guessing or not using the statistical analysis named guessing analysis andpersonfitanalysis. 5. Inconsistency score from different raters 5. Appointtheratertomarkcertain questionsforalllearners(Thisalways happen when you have more than onesectionandhavemorethanone lecturer). CONCLUSION Coming back to the issue of validity and reliability in assessment, there is a need for educators to put an effort to ensurethattheitemsintheformofquestionsorinstructions arenotonlyclearbutalsoabletomeasurewhatitisintended tomeasurebasedontherelatedlearningoutcomes. Establishingvalidityandreliabilityofinstrumentscan provide educators with some indications of the quality of the measuring tools being used. Valid and reliable instruments enabletheeducatorstocontinuouslyusethemeasuringtools withoutreservation. Reliablenot valid Precisenot Accurate Reliableand valid Preciseand Accurate NotReliable butvalid NotPrecisebut Accurate NotReliable butNotvalid NotPrecisebut NotAccurate 94 ". Would you like to continue making it or start afresh?

“There’s No Such Thing as Sound Science” by By Christie Aschwanden was a lead science writer for FiveThirtyEight. FiveThirtyEight, Science, Dec. 6, 2017 Science is being turned against itself. For decades, its twin ideals of transparency and rigor have been weaponized by those who disagree with results produced by the scientific method. Under the Trump administration, that fight has ramped up again. In a move ostensibly meant to reduce conflicts of interest, Environmental Protection Agency Administrator Scott Pruitt has removed a number of scientists from advisory panels and replaced some of them with representatives from industries that the agency regulates. Like many in the Trump administration, Pruitt has also cast doubt on the reliability of climate science. For instance, in an interview with CNBC, Pruitt said that “measuring with precision human activity on the climate is something very challenging to do.” Similarly, Trump’s pick to head NASA, an agency that oversees a large portion the nation’s climate research, has insisted that research into human influence on climate lacks certainty, and he falsely claimed that “global temperatures stopped rising 10 years ago.” Kathleen Hartnett White, Trump’s nominee to head the White House Council on Environmental Quality, said in a Senate hearing last month that she thinks we “need to have more precise explanations of the human role and the natural role” in climate change. The same entreaties crop up again and again: We need to root out conflicts. We need more precise evidence. What makes these arguments so powerful is that they sound quite similar to the points raised by proponents of a very different call for change that’s coming from within science. This other movement strives to produce more robust, reproducible findings. Despite having dissimilar goals, the two forces espouse principles that look surprisingly alike: Science needs to be transparent. Results and methods should be openly shared so that outside researchers can independently reproduce and validate them. The methods used to collect and analyze data should be rigorous and clear, and conclusions must be supported by evidence. These are the arguments underlying an “open science” reform movement that was created, in part, as a response to a “reproducibility crisis” that has struck some fields of science.1 But they’re also used as talking points by politicians who are working to make it more difficult for the EPA and other federal agencies to use science in their regulatory decision-making, under the guise of basing policy on “sound science.” Science’s virtues are being wielded against it. What distinguishes the two calls for transparency is intent: Whereas the “open science” movement aims to make science more reliable, reproducible and robust, proponents of “sound science” have historically worked to amplify uncertainty, create doubt and undermine scientific discoveries that threaten their interests. “Our criticisms are founded in a confidence in science,” said Steven Goodman, co-director of the Meta-Research Innovation Center at Stanford and a proponent of open science. “That’s a fundamental difference — we’re critiquing science to make it better. Others are critiquing it to devalue the approach itself.” Calls to base public policy on “sound science” seem unassailable if you don’t know the term’s history. The phrase was adopted by the tobacco industry in the 1990s to counteract mounting evidence linking secondhand smoke to cancer. A 1992 Environmental Protection Agency report identified secondhand smoke as a human carcinogen, and Philip Morris responded by launching an initiative to promote what it called “sound science.” In an internal memo, Philip Morris vice president of corporate affairs Ellen Merlo wrote that the program was designed to “discredit the EPA report,” “prevent states and cities, as well as businesses from passing smoking bans” and “proactively” pass legislation to help their cause. The sound science tactic exploits a fundamental feature of the scientific process: Science does not produce absolute certainty. Contrary to how it’s sometimes represented to the public, science is not a magic wand that turns everything it touches to truth. Instead, it’s a process of uncertainty reduction, much like a game of 20 Questions. Any given study can rarely answer more than one question at a time, and each study usually raises a bunch of new questions in the process of answering old ones. “Science is a process rather than an answer,” said psychologist Alison Ledgerwood of the University of California, Davis. Every answer is provisional and subject to change in the face of new evidence. It’s not entirely correct to say that “this study proves this fact,” Ledgerwood said. “We should be talking instead about how science increases or decreases our confidence in something.” The tobacco industry’s brilliant tactic was to turn this baked-in uncertainty against the scientific enterprise itself. While insisting that they merely wanted to ensure that public policy was based on sound science, tobacco companies defined the term in a way that ensured that no science could ever be sound enough. The only sound science was certain science, which is an impossible standard to achieve. “Doubt is our product,” wrote one employee of the Brown & Williamson tobacco company in a 1969 internal memo. The note went on to say that doubt “is the best means of competing with the ‘body of fact’” and “establishing a controversy.” These strategies for undermining inconvenient science were so effective that they’ve served as a sort of playbook for industry interests ever since, said Stanford University science historian Robert Proctor. The sound science push is no longer just Philip Morris sowing doubt about the links between cigarettes and cancer. It’s also a 1998 action plan by the American Petroleum Institute, Chevron and Exxon Mobil to “install uncertainty” about the link between greenhouse gas emissions and climate change. It’s industry-funded groups’ late-1990s effort to question the science the EPA was using to set fine-particle-pollution air-quality standards that the industry didn’t want. And then there was the more recent effort by Dow Chemical to insist on more scientific certainty before banning a pesticide that the EPA’s scientists had deemed risky to children. Now comes a move by the Trump administration’s EPA to repeal a 2015 rule on wetlands protection by disregarding particular studies. (To name just a few examples.) Doubt merchants aren’t pushing for knowledge, they’re practicing what Proctor has dubbed “agnogenesis” — the intentional manufacture of ignorance. This ignorance isn’t simply the absence of knowing something; it’s a lack of comprehension deliberately created by agents who don’t want you to know, Proctor said.2 In the hands of doubt-makers, transparency becomes a rhetorical move. “It’s really difficult as a scientist or policy maker to make a stand against transparency and openness, because well, who would be against it?” said Karen Levy, researcher on information science at Cornell University. But at the same time, “you can couch everything in the language of transparency and it becomes a powerful weapon.” For instance, when the EPA was preparing to set new limits on particulate pollution in the 1990s, industry groups pushed back against the research and demanded access to primary data (including records that researchers had promised participants would remain confidential) and a reanalysis of the evidence. Their calls succeeded and a new analysis was performed. The reanalysis essentially confirmed the original conclusions, but the process of conducting it delayed the implementation of regulations and cost researchers time and money. Delay is a time-tested strategy. “Gridlock is the greatest friend a global warming skeptic has,” said Marc Morano, a prominent critic of global warming research and the executive director of ClimateDepot.com, in the documentary “Merchants of Doubt” (based on the book by the same name). Morano’s site is a project of the Committee for a Constructive Tomorrow, which has received funding from the oil and gas industry. “We’re the negative force. We’re just trying to stop stuff.” Some of these ploys are getting a fresh boost from Congress. The Data Quality Act (also known as the Information Quality Act) was reportedly written by an industry lobbyist and quietly passed as part of an appropriations bill in 2000. The rule mandates that federal agencies ensure the “quality, objectivity, utility, and integrity of information” that they disseminate, though it does little to define what these terms mean. The law also provides a mechanism for citizens and groups to challenge information that they deem inaccurate, including science that they disagree with. “It was passed in this very quiet way with no explicit debate about it — that should tell you a lot about the real goals,” Levy said. But what’s most telling about the Data Quality Act is how it’s been used, Levy said. A 2004 Washington Post analysis found that in the 20 months following its implementation, the act was repeatedly used by industry groups to push back against proposed regulations and bog down the decision-making process. Instead of deploying transparency as a fundamental principle that applies to all science, these interests have used transparency as a weapon to attack very particular findings that they would like to eradicate. Now Congress is considering another way to legislate how science is used. The Honest Act, a bill sponsored by Rep. Lamar Smith of Texas,3 is another example of what Levy calls a “Trojan horse” law that uses the language of transparency as a cover to achieve other political goals. Smith’s legislation would severely limit the kind of evidence the EPA could use for decision-making. Only studies whose raw data and computer codes were publicly available would be allowed for consideration. That might sound perfectly reasonable, and in many cases it is, Goodman said. But sometimes there are good reasons why researchers can’t conform to these rules, like when the data contains confidential or sensitive medical information.4 Critics, which include more than a dozen scientific organizations, argue that, in practice, the rules would prevent many studies from being considered in EPA reviews.5 It might seem like an easy task to sort good science from bad, but in reality it’s not so simple. “There’s a misplaced idea that we can definitively distinguish the good from the not-good science, but it’s all a matter of degree,” said Brian Nosek, executive director of the Center for Open Science. “There is no perfect study.” Requiring regulators to wait until they have (nonexistent) perfect evidence is essentially “a way of saying, ‘We don’t want to use evidence for our decision-making,’” Nosek said. Most scientific controversies aren’t about science at all, and once the sides are drawn, more data is unlikely to bring opponents into agreement. Michael Carolan, who researches the sociology of technology and scientific knowledge at Colorado State University, wrote in a 2008 paper about why objective knowledge is not enough to resolve environmental controversies. “While these controversies may appear on the surface to rest on disputed questions of fact, beneath often reside differing positions of value; values that can give shape to differing understandings of what ‘the facts’ are.” What’s needed in these cases isn’t more or better science, but mechanisms to bring those hidden values to the forefront of the discussion so that they can be debated transparently. “As long as we continue down this unabashedly naive road about what science is, and what it is capable of doing, we will continue to fail to reach any sort of meaningful consensus on these matters,” Carolan writes. The dispute over tobacco was never about the science of cigarettes’ link to cancer. It was about whether companies have the right to sell dangerous products and, if so, what obligations they have to the consumers who purchased them. Similarly, the debate over climate change isn’t about whether our planet is heating, but about how much responsibility each country and person bears for stopping it. While researching her book “Merchants of Doubt,” science historian Naomi Oreskes found that some of the same people who were defending the tobacco industry as scientific experts were also receiving industry money to deny the role of human activity in global warming. What these issues had in common, she realized, was that they all involved the need for government action. “None of this is about the science. All of this is a political debate about the role of government,” she said in the documentary. These controversies are really about values, not scientific facts, and acknowledging that would allow us to have more truthful and productive debates. What would that look like in practice? Instead of cherry-picking evidence to support a particular view (and insisting that the science points to a desired action), the various sides could lay out the values they are using to assess the evidence. For instance, in Europe, many decisions are guided by the precautionary principle — a system that values caution in the face of uncertainty and says that when the risks are unclear, it should be up to industries to show that their products and processes are not harmful, rather than requiring the government to prove that they are harmful before they can be regulated. By contrast, U.S. agencies tend to wait for strong evidence of harm before issuing regulations. Both approaches have critics, but the difference between them comes down to priorities: Is it better to exercise caution at the risk of burdening companies and perhaps the economy, or is it more important to avoid potential economic downsides even if it means that sometimes a harmful product or industrial process goes unregulated? In other words, under what circumstances do we agree to act on a risk? How certain do we need to be that the risk is real, and how many people would need to be at risk, and how costly is it to reduce that risk? Those are moral questions, not scientific ones, and openly discussing and identifying these kinds of judgment calls would lead to a more honest debate. Science matters, and we need to do it as rigorously as possible. But science can’t tell us how risky is too risky to allow products like cigarettes or potentially harmful pesticides to be sold — those are value judgements that only humans can make.

Steps in Thematic Analysis.Step 4: Reviewing themes Now we have to make sure that our themes are useful and accurate representations of the data. Here, we return to the data set and compare our themes against it. Are we missing anything? Are these themes really present in the data? What can we change to make our themes work better? If we encounter problems with our themes, we might split them up, combine them, discard them or create new ones: whatever makes them more useful and accurate. For example, we might decide upon looking through the data that “changing terminology” fits better under the “uncertainty” theme than under “distrust of experts,” since the data labelled with this code involves confusion, not necessarily distrust. Step 5: Defining and naming themes Now that you have a final list of themes, it’s time to name and define each of them. Defining themes involves formulating exactly what we mean by each theme and figuring out how it helps us understand the data. Naming themes involves coming up with a succinct and easily understandable name for each theme. For example, we might look at “distrust of experts” and determine exactly who we mean by “experts” in this theme. We might decide that a better name for the theme is “distrust of authority” or “conspiracy thinking”. Step 6: Writing up Finally, we’ll write up our analysis of the data. Like all academic texts, writing up a thematic analysis requires an introduction to establish our research question, aims and approach. We should also include a methodology section, describing how we collected the data (e.g. through semi-structured interviews or open-ended survey questions) and explaining how we conducted the thematic analysis itself. The results or findings section usually addresses each theme in turn. We describe how often the themes come up and what they mean, including examples from the data as evidence. Finally, our conclusion explains the main takeaways and shows how the analysis has answered our research question. In our example, we might argue that conspiracy thinking about climate change is widespread among older conservative voters, point out the uncertainty with which many voters view the issue, and discuss the role of misinformation in respondents’ perceptions.

Interview Questions

DM Interview questions

Related quizzes

Related quizzes