 |
J. Scott Armstrong
· Faculty Page
· Major Findings
· Educational Materials
Selected Papers
· Applied Statistics· Communications
· Education
· Forecasting
· Implementing Change
· Marketing
· Scientific Method
· Social Responsibility
· Strategy and Planning
People
· Faculty
· Former Faculty
· PhD Students
· Phd Alumni
· MBA Students
· Undergraduates
· Staff
Home
|
|
|
|
Selected Published
Books and Papers BOOKS
| |
J. Scott Armstrong (2007), Persuasive Advertising. An
evidence-based approach for developing advertisements.
Forthcoming from Palgrave Macmillan. Draft copy available upon
request.
|
FORTHCOMING
 |
| |
Principles of Forecasting: A
Handbook for Researchers and Practitioners (2001, Kluwer
Academic Publishers) A book designed to summarize all knowledge
about forecasting methods; the primary emphasis is on empirical
findings, but reliance is also placed on expert opinion.
|
 |
| |
Long-Range
Forecasting, 2nd. Ed. (1985, John Wiley & Sons)
The only comprehensive summary of
research on forecasting prior to 1985, it covers judgmental and quantitative methods,
and it has stood the test of
time. The book is out of print but used copies and
reviews are available at
amazon.com. |
 |
PAPERS (in PDF format)
Raymond Hubbard and J. Scott Armstrong, "Why
We Don't Really Know What 'Statistical Significance' Means: A Major
Educational Failure"
J. Scott Armstrong (1970), "How to Avoid Exploratory
Research," Journal of Advertising Research, 10 (4),
27-30 Stepwise regressions led to false
conclusions. Moral: use theory not statistics to select variables.
J. Scott Armstrong and Peer Soelberg, (1968), "On the
Interpretation of Factor Analysis," Psychological Bulletin,
70 (5), 361-364
Theory should precede, not follow the
use of factor analysis.
J. Scott Armstrong. (1967), "Derivation
of Theory by Means of Factor Analysis, or Tom Swift and His
Electric Factor Analysis Machine," The American
Statistician (December), 17-21
Stepwise regression is a poor way to discover
underlying structure. This paper is relevant to the current approach
called "data mining."
Return to top
J. Scott Armstrong (1978), "The Graffiti Solution," The Wharton
Magazine, Winter, 12-14 - Want to stop graffiti? Pay people to do it
but hire few people. And, you know, Philadelphia has done this.
Return to top
J. Scott Armstrong (2004), "Improving Learning
at Universities: Who is Responsible?" University of Pennsylvania
Almanac, Vol. 51, No. 15, December 14, 2004 Contains evidence-based suggestions for
making students responsible for their learning.
J. Scott Armstrong (1998), "Are Student Ratings of Instruction
Useful?" American Psychologist, 53 (November), 1223-1224 Teacher ratings have been shown to be unrelated
to traditional learning measures. It seems likely that they are
detrimental to the learning of skills. Direct measures of learning
should be used, not those based on liking the teacher.
J. Scott Armstrong (1995), "The Devil's Advocate Responds to an MBA
Student's Claim that Research Harms Learning," Journal of
Marketing, 59 (July), 101-106
Faculty who are engaged more heavily in research spend
less time in teaching, yet they do not receive lower teacher
evaluations. Their students seem to learn more. Also, graduates of
schools that produce more research earn more in the five years after
graduation.
J. Scott Armstrong and Tad Sperry (1994), "Business School
Prestige Research versus Teaching," Interfaces, 24
(2), 13-43 - The reputations of business schools were
strongly related to research, even when one considers only the top
schools. Teacher evaluations were not related to business school
prestige.
J. Scott Armstrong (1994), "Would
Mandatory Attendance be Effective for Economics Classes?" Internet
publication only - Based on prior empirical studies,
mandatory attendance would not help learning - a conclusion that goes beyond
economics classes.
J. Scott Armstrong (1990), "The
Case for Minimum Teaching Standards," Almanac
(University of Pennsylvania), January 16, pp. 9-11 - A review of empirical evidence suggests
that an emphasis on teaching will harm learning.
J. Scott Armstrong (1983), "Learner Responsibility in Management
Education, or Ventures into Forbidden Research" with comments, Interfaces,
13, 26-38 - Learning is highly dependent on the
learner taking responsibility. Grading leads to a loss of
responsibility. This paper reports on evidence showing that the use of
time-contracts (grades based only on learning time) led to higher
student responsibility and increased learning.
J. Scott Armstrong (1980), "Teacher
vs. Learner Responsibility in Management Education," Working Paper - Provides empirical evidence showing that
learning depends on learner responsibility.
J. Scott Armstrong (1979), "The Natural Learning Project," Journal
of Experimental Learning and Simulation, 1, 5-12 Reports on five learning experiments in business
school courses. Self-oriented skill training was more
than three times as effective as traditional courses in producing useful
behavioral changes.
J. Scott Armstrong (1977), "Designing and Using Experiential Exercises," in
Experiential Learning in Marketing Education, M. D. DeLozier, D.M.
Lewison, and R. Andress (eds.), Occasional Studies No. 11, March,
Division of Research, College of Business Administration, University of
South Carolina, pp. 8-17.
Full Text (PDF) - Describes evidence showing how to
design experiential learning.
Return to top
J. Scott Armstrong (2005), "The Forecasting Canon: Nine
Generalizations to Improve Forecast Accuracy," Foresight: The
International Journal of Applied Forecasting, 1 (1), June, 29-35.
Kesten Green and J. Scott Armstrong (2005), "Structured
Analogies for Forecasting"
- The use of the structured analogies procedure led to a
41% error reduction in forecasting the decisions in eight conflict
situations.
J. Scott Armstrong, Fred Collopy, and J. Thomas Yokum
(2005), "Decomposition by Causal Forces: A Procedure for Forecasting
Complex Time Series," International Journal of Forecasting, 21,
25-36.
For nine series in which conditions
were met, decomposition of time series by causal forces reduced forecast
error by more than half.
J. Scott Armstrong (2004), "Damped Seasonality Factors:
Introduction," International Journal of Forecasting, 20,
2004, 523-527 - Introduction to a special section on the
value of damping seasonality estimates when uncertainty is encountered.
J. Scott Armstrong (2003), Review of Predicting Presidential
Elections and Other Things, (Ray Fair, Stanford University Press;
Stanford, CA 2002), International Journal of Forecasting,
19,
760-761 - Shows how to use econometric models to
forecast such practical things as marital happiness, how attendance at
classes relates to grades, and how fast you can run the marathon.
Unfortunately, Fair's models do not help much with respect to policy
changes, the area where econometric models are most advantageous.
J. Scott Armstrong (2002), "Assessing Game Theory, Role Playing, and Unaided
Judgment," International Journal of Forecasting, 18 (3), 345-352
Unaided judgment and game theorists were
unable to provide useful forecasts of decisions made in conflicts. In
contrast, simulated interactions (a type of role playing) were quite
accurate.
Written as a response to: Kesten C. Green, "Forecasting decisions in conflict
situations: A comparison of game theory, role-playing, and
unaided judgment," International Journal of
Forecasting, 18, 321-344
Full Text.
See also Green's reply to commentators, "Embroiled in a
conflict: Who do you call?" International Journal of
Forecasting, 28, 389-395 -
Full Text
J. Scott Armstrong (2002),
Review of Batchelor, Roy (2001), "How useful are the forecasts of
intergovernmental agencies? The IMF and OECD versus the consensus,"
(Applied Econometrics, 33, 225-235), International Journal of
Forecasting, 18, 482-483
J. Scott Armstrong (2001), "Should We Redesign Forecasting
Competitions?" International Journal of Forecasting, 17,
542-545 - In the future, competitions should
start with hypotheses as to which methods will be most effective
under what conditions. In addition, they should allow for the use of domain knowledge.
Monica Adya, Fred Collopy, J. Scott Armstrong and Miles Kennedy
(2001), "Automatic Identification of Time Series Features for Rule-Based
Forecasting," International Journal of Forecasting, 17,
143-157 - Automatic procedures, which are less
expensive and more reliable than judgmental procedures, produced
rule-based forecasts with little loss in forecast accuracy.
J. Scott Armstrong and Fred Collopy (2001),
"Identification of Asymmetric Prediction Intervals through Causal
Forces," Journal of Forecasting, 20, 273-283 When forecast errors are large, as
is common in annual forecasting, errors are asymmetrical in
percentage terms. Log transformations can correct for this asymmetry,
although asymmetry in the logs occurs for "contrary" time series.
J. Scott Armstrong and Thomas Yokum (2001), "Potential
Diffusion of
Expert Systems in Forecasting: An Application of Diffusion Theory," Technological Forecasting and Social
Change, 67, 95-105; Summary
review by
Monica Adya - Surveys of researchers,
educators, practitioners, and decision makers revealed high perceived
risk for the adoption of expert systems.
J. Scott Armstrong, Vicki G. Morwitz and V, Kumar (2000), "Sales Forecasts for
Existing Consumer Products and Services: Do Purchase Intentions Contribute to
Accuracy?" International Journal of Forecasting, 16
383-397
- Intentions data are useful even when one has historical
time series. In the study of consumer durables, intentions led to a 1/3
reduction in forecast errors. This paper was named as one of four
outstanding papers published in the International Journal of
Forecasting for the period 2000-2001."
Monica Adya, J. Scott Armstrong, Fred Collopy, & Miles Kennedy (2000), "An
Application of Rule-based Forecasting for a Situation Lacking
Domain Knowledge," International Journal of
Forecasting, 16, 477-484 This paper was part of the
M3-competition. A simplified version of RBF performed well without using
domain knowledge.
Armstrong, J. Scott (1999),
"Sales Forecasting,"
in The IEBM Encyclopedia of Marketing, Michael J. Baker
(Ed.), London, International Thomson Business Press, 278-290.
J. Scott Armstrong (1999), "Forecasting for Environmental
Decision-Making," in V.H. Dale and M.E. English, eds., Tools to Aid
Environmental Decision Making, New York: Springer-Verlag, pp. 192-225
J. Scott Armstrong and R. J. Brodie (1999),
"Forecasting for Marketing," in G. J. Hooley and M. K. Hussey
(eds.), Quantitative Methods in Marketing, 2nd ed., London:
International Thompson Business press, pp. 92-119 - This
review of empirical evidence leads to recommendations on how to improve
forecast accuracy in marketing.
J. Scott Armstrong (1998), "Commentaries on 'Generalizing about Univariate Forecasting Methods: Further Empirical Evidence,'"
International Journal of Forecasting, 14, 359-366 Comparative studies such as the M-competitions
should fully describe the conditions that describe the data.
J. Scott Armstrong and Fred Collopy (1998), "Integration of Statistical
Methods and Judgment for Time Series Forecasting: Principles from
Empirical Research," in G. Wright and P. Goodwin (eds.),
Forecasting with Judgment. John Wiley & Sons Ltd., 269-293, with
review by N. R.
Sanders, International Journal of Forecasting, 15 (1999),
345-346 Presents all feasible ways to
combine judgment and statistical forecasts and reviews empirical
evidence on each.
J. Scott Armstrong and Robert Fildes (1995), "On the Selection
of Error Measures for Comparisons Among Forecasting Methods," Journal
of Forecasting, 14, 67-71 This review of empirical evidence leads
to the conclusion that mean square errors are highly unreliable and
should not be used in forecasting.
J. Thomas Yokum and J. Scott Armstrong (1995),
"Beyond Accuracy: Comparison of Criteria Used to Select Forecasting
Methods," International Journal of Forecasting, 11,
591-597 According to experts, accuracy is always important, yet other criteria,
such as ease of use,
are nearly as important. The importance of criteria varied depending on
the type of forecasting problem.
J. Scott Armstrong and Fred Collopy (1994), "How Serious are
Methodological Issues in Surveys: A Reexamination of the Clarence Thomas Polls,"
Internet publication. Decisions can be improved if criteria are
established before making a decision. Once a decision was made, decision
makers were reluctant to change even if their decision violates their own
criteria.
Fred Collopy, Monica Adya & J. Scott Armstrong (1994), "Principles
for Examining Predictive Validity: The Case of Information
Systems Spending Forecasts," Information Systems
Research, 5, 170-179 Demonstrates the use of
principles for evaluating forecasting methods. This paper shows how
a previously published paper falsely concluded that a diffusion
model was useful in forecasting spending on information systems. In
fact, a single linear trend extrapolation produced more
accurate forecasts than the diffusion model.
J. Scott Armstrong (1994), "The
Fertile Field of Meta-Analysis: Cumulative Progress in
Agricultural Forecasting," International Journal
of Forecasting, 10, 147-149
MacGregor, D. & J. Scott Armstrong
(1994), "Judgmental
Decomposition: When Does It Work?" International
Journal of Forecasting, 10, 495-506 - Decomposition is useful when dealing
with large numbers (or very small numbers) and when one knows more about
the parts than the whole. Otherwise, it leads to increased errors.
J. Scott Armstrong and Fred Collopy (1993), "Causal
Forces: Structuring Knowledge for Time-series Extrapolation," Journal
of Forecasting, 12, 103-115 Domain knowledge, expressed as expectations about
trends in time series, led to improved accuracy in time-series
forecasts.
J. Scott Armstrong and Fred Collopy (1992), "Error Measures for
Generalizing about Forecasting Methods: Empirical Comparisons," International
Journal of Forecasting, 8, 69-80 Proposes the Relative Absolute Error
(RAE) and compares
it with other error measures for comparisons among time-series methods.
It is reliable and easy to interpret. A commentary on this paper was
followed by our reply:
Fred Collopy and J. Scott Armstrong (1992), "Expert Opinions
about Extrapolation and the Mystery of the Overlooked
Discontinuities," International Journal of Forecasting, 8,
575-582 - Traditional time-series extrapolative
methods do not perform well when there are discontinuities in the data,
according to this survey of forecasting experts.
Fred Collopy and J. Scott Armstrong (1992), "Rule-Based
Forecasting: Development and Validation of an Expert Systems Approach to
Combining Time Series Extrapolations," Management Science,
38 (10), 1394-1414 Uses prior knowledge about forecasting
methods and domain knowledge to formulate rules for time series
forecasting.
J. Scott Armstrong (1991), "Prediction of Consumer Behavior by
Experts and Novices," Journal of Consumer Research, 18
(September), 251-256 Academics who are familiar with research on consumer
behavior were no more accurate than practitioners or high school
students when making forecasts about the outcomes of studies on consumer
behavior.
Fred Collopy and J. Scott Armstrong (1989), "Toward
Computer-Aided Forecasting Systems: Gathering, Coding, and Validating
the Knowledge," in George R. Widmeyer (ed.), DSS-899
Transactions: Ninth International Conference on Decision Support Systems,
Institute of Management Science, pp. 103-119 Describes how to develop rules for
forecasting.
Stephen Dakin and J. Scott Armstrong (1989), "Predicting
Job Performance: A Comparison of Expert Opinion and Research
Findings," International Journal of Forecasting,
5, 187-194 Despite a wealth of useful
research on personnel selection, practitioners rely on invalid
procedures.
J. Scott Armstrong (1989), "Combining
Forecasts: The End of the Beginning or the Beginning of the
End?" International Journal of Forecasting, 5,
585-588
J. Scott Armstrong (1988), "Review of Ravi Batra, The Great Depression
of 1990," International
Journal of Forecasting, 4, 493-502 Identifies faulty forecasting procedures
in a popular book.
J. Scott Armstrong (1988),
"Communication
of Research in Forecasting: The Journal," International
Journal of Forecasting, 4, 321-324
Reports on a survey of 201 readers of two forecasting journals. A
substantial amount of findings were used by practitioners and
researchers.
J. Scott Armstrong (1987), "Forecasting
Methods for Conflict Situations," in G. Wright and P. Ayton (eds.), Judgmental Forecasting,
pp. 157-176 For a more recent report, see the
Role Playing chapter in
Principles of Forecasting.
J. Scott Armstrong, R. Brodie & S. McIntyre (1987), "Forecasting
Methods for Marketing: Review of Empirical Research," International Journal of
Forecasting, 3, 335-376
J. Scott Armstrong (1986), "Research on
Forecasting: A Quarter-Century Review, 1960-1984," Interfaces,
16, 1 (Jan-Feb), 89-109 - This review shows that substantial
progress has been made since 1960. That said, much effort is still
being devoted to areas that have little promise, while some highly
promising areas are ignored.
J. Scott Armstrong (1986),
"Review
of Steven J. Rosenstone, Forecasting Presidential Elections,"
International Journal of Forecasting, 2, 248-249.
This book used causal models to forecast U.S.
presidential votes by state, then aggregates across states. It tested
well in three elections.
J. Scott Armstrong (1984),
"Forecasting
by Extrapolation: Conclusions from 25 Years of Research," Interfaces,
14 (Nov.-Dec.), 52-66, with commentaries and reply. Relatively simple
extrapolation methods were found to be more accurate than sophisticated
methods.
J. Scott Armstrong (1984),
Do Judgmental Researchers Use Their Own Research?
A Review of
Judgment Under Uncertainty: Heuristics and Biases,
Journal of Forecasting, 3 (1984), 235 239.
J. Scott Armstrong (1983), "Relative
Accuracy of Judgmental and Extrapolative Methods in
Forecasting Annual Earnings," Journal of
Forecasting, 2, 437-447 Management forecasts of firms' annual earnings
were more accurate than those by professional independent analysts, and
these were more accurate than statistical extrapolations.
J. Scott Armstrong & Ed Lusk (1983),
"Commentary
on the Makridakis Time Series Competition
(M-Competition)," with comments by Gardner, Geurts,
Lopes, Markland, McLaughlin, Newbold, Pack, and replies by
Andersen, Carbone, Fildes, Parzen, Newton, Winkler, & Makridakis, Journal of
Forecasting 2, 259-311. Provides an
introduction to the commentaries on the M-competition.
Robert Carbone and J. Scott Armstrong (1982), "Evaluation of
Extrapolative Forecasting Methods: Results of a Survey of Academicians and
Practitioners," Journal of Forecasting 1, 215-217 Reports on a survey of academics
and practitioners dealing with beliefs about forecasting methods. For
example, they favored the RMSE for evaluation.
J. Scott Armstrong (1981), "How Expert Are the Experts?"
Inc., December, 15-16 - A simpler version of the paper from
the Technology Review (1980).
J. Scott Armstrong (1980), "The Seer-Sucker Theory: The Value of
Experts in Forecasting," Technology Review, June/July,
16-24
Expertise above a minimal level does little to improve
the accuracy for forecasts of changes over time.
J. Scott Armstrong (1978a), "Forecasting
with Econometric Methods," Journal of Business, 51 (4),
549-564
- This study, using a survey of the world's leading
econometricians, found conflicts between the econometricians' procedures
and results from empirical studies. The paper was published along with
commentaries by econometricians. This was followed by the next paper.
J. Scott Armstrong (1978b), "Econometric Forecasting and the
Science Court," Journal of Business, 51 (4), 595-600
J. Scott Armstrong (1975), "Tom
Swift and His Electric Regression Analysis Machine: 1973," Psychological
Reports, 36, 806 Illustrates problems with non-theoretical
statistical analyses by reporting on a published study that used
stepwise regression with 115 variables and 19 observations.
J. Scott Armstrong and Alan C. Shapiro (1974), "Analyzing Quantitative
Models," Journal of Marketing, 38, 61-66 We present a framework for analyzing models, and illustrate
its use in analyzing a popular marketing model.
J. Scott Armstrong & M. Grohman (1972), "A
Comparative Study of Methods for Long-Range Market
Forecasting," Management Science, 19, 211-221 - Econometric methods were more
accurate than extrapolation methods.
J. Scott Armstrong (1971), "Exploratory Analysis of Marketing Data: A
Reply" [to Douglass C. Crocker, Journal of Marketing Research, 8 (1971) 509-510], Journal
of Marketing Research, 8, 511-513
J. Scott Armstrong and T. Overton (1971), "Brief
vs. Comprehensive Descriptions in Measuring Intentions to Purchase,"
Journal of Marketing Research, 8 (February), 114-117 Brief descriptions of new products
produced similar estimates for intention to purchase as
elaborate and expensive descriptions.
J. Scott Armstrong (1970), "An Application of Econometric Models to
International Marketing," Journal of Marketing Research,
7, 190-198 Econometric methods produced substantially
more accurate long-range forecasts of the photographic markets in 17
countries than did extrapolations.
J. Scott Armstrong and James G. Andress (1970), "Exploratory Analysis of
Marketing Data: Trees vs. Regression," Journal of Marketing
Research, 7, 487-492 Tree analysis (using Automatic Interaction
Detector) was more accurate than regression in forecasting the sales at
2,717 gas stations. With "A Reply to Cocker," Journal of
Marketing Research, 8 (1971), 511-513.
J. Scott Armstrong and John U. Farley (1969), "A Note on the Use of
Markov Chains in Forecasting Store Choice," Management Science,
16 (4) - Markov chains proved to be of little
value in forecasting customers' choice of a food store.
Return to top
J. Scott Armstrong (1983), "Review of Intentional Changes: A
Fresh Approach to Helping People Change, by Allen Tough," Academy
of Management Review 8, 509-511 Groups, especially those with an
appointed leader, inhibit learning.
- J. Scott Armstrong (1982), "Strategies for Implementing Change:
An Experiential Approach," Group and Organization Studies,
7 (4), 457-475 - Describes the "Delta" approach to
implementing change in organizations. The key is to gain agreement
on the design of a small experiment, rather than to try to sell the
change itself.
Return to top
J. Scott Armstrong and Kesten C. Green (2005), "Demand
Forecasting - Evidence-Based Methods," Working paper.
J. Scott Armstrong (2003), "How to be Less Persuaded or More Persuasive Review
of Age of Propaganda: The Everyday Use and Abuse of Persuasion,"
by Anthony R. Prakanis and Elliot Aronson, Journal of Marketing,
67 (1), 129-130
- This review illustrates the use of some common but
non-obvious attempts to persuade.
J. Scott Armstrong (2003), "Discovery and Communication of
Important Marketing Findings: Evidence and Proposals,"
Journal of Business Research, 56, 69-84, with commentaries by
Rossiter and Lehmann, 85-90.
Response to commentary published as "The Value of Surprising Findings
for Research on Marketing," Journal of Business Research,
2003, 91-92
A review of empirical evidence
shows that current procedures for journal publication inhibit
scientific progress. This paper suggests ways to improve the process.
Raymond Hubbard and J. Scott Armstrong (1997), "Publication Bias
Against Null Results," Psychological Reports, 80,
337-338 - In some fields, such as psychology, well over
90% of papers reject the null hypotheses. This is unfortunate
because many papers with null results provide useful findings.
- J. Scott Armstrong (1996), "How Should Firms Select Advertising
Agencies? A Review of Randall Rothenberg's Where the Suckers Moon," Journal of Marketing,
60, 131-134 - This paper describes the process
used to select an advertising agency for Subaru America. It then
describes a structured approach to improve the selection of an
agency.
J. Scott Armstrong (1994), "Review of Scott Plous, The Psychology of Judgment and Decision
Making," Journal
of Marketing, 58 (July)
J. Scott Armstrong and Roderick J. Brodie (1994), "Effects
of Portfolio Planning Methods on Decision Making: Experimental
Results," International Journal of Research in Marketing,
11, 73-84 Portfolio planning methods, such as the BCG
matrix, harm decision making.
J. Scott Armstrong and Roderick J. Brodie (1994), "Portfolio
Planning Methods: Faulty Approach or Faulty Research? A Rejoinder
to Making Better Decisions by Wensley," International
Journal of Research in Marketing, 11, 91-93
J. Scott Armstrong and Fred Collopy (1994), "The Profitability of Winning,"
Chief Executive, June, 1994, 61-63 - A discussion of research on the
myth of market share. This was aimed at executives.
J. Scott Armstrong and J. Thomas Yokum (1994), "Effectiveness of
Monetary Incentives: Mail Surveys to Members of Multinational
Professional Groups," Industrial Marketing Management,
23, 133-136 - This review of experimental
studies shows that small monetary incentives always improve response
rates, even for the situation we examined, professional groups, where their use would seem odd.
J. Scott Armstrong, Nicole Coviello, and Barbara Safranek (1993),
"Escalation Bias: Does It Extend to Marketing?" Journal of the
Academy of Marketing Science, 21 (2), 247-253
- Escalation bias did not harm decision-making in
marketing. Furthermore, prior studies on escalation bias did not
demonstrate that it harmed decision making.
J. Scott Armstrong and Randall L. Schultz (1993), "Principles Involving
Marketing Policies: An Empirical Assessment," Marketing Letters,
4:3, 253-265 - We examined nine marketing textbooks,
published since 1927, to see if they contained useful marketing
principles. Four doctoral students found 566 normative statements
about pricing, product, place, or promotion in these texts. None of
these statements were supported by empirical evidence.
Raymond Hubbard and J. Scott Armstrong (1992), "Are Null Results Becoming
an Endangered Species in Marketing?" Marketing Letters,
3:2, 127-136 - Of 692 papers using statistical
significance between 1974 and 1989 in the three major marketing
journals, fewer than 8% led to a rejections of the null hypothesis.
This percentage dropped by 1/2 from the 1970s to the 1980s. This is
an unfortunate situation.
J. Scott Armstrong (1990), "Class of Mail Does Affect Response
Rates to Mailed Questionnaires: Evidence from Meta-Analysis,"
Journal of the Market Research Society, 32 (3), 469-472, with a
reply by Lee Harvey - U.S. business reply postage should not be used
in survey research.
J. Scott Armstrong and Philip D. Hutcherson (1989),
"Predicting the Outcome of Marketing Negotiations: Role-Playing
Versus Unaided Opinions," International Journal of Research in
Marketing, 6, 227-239 Role playing was much more accurate
than unaided judgment in predicting decisions made in negotiations.
J. Scott Armstrong and Edward J. Lusk (1987), "Return Postage
in Mail Surveys: A Meta-Analysis," Public Opinion Quarterly,
51 (2), 233-248. Full
Text (PDF) - Real stamps improve the return rate.
Surveys using business reply postage are less effective.
J. Scott Armstrong (1978), "The Manager's Dilemma: Role Conflict in
Marketing," in Future Directions for Marketing, G. Fisk, J.
Arndt, and G. Gronhaug (eds.), Cambridge, MA: Marketing Science
Institute, pp. 78-89.
Full Text (PDF)
J. Scott Armstrong (1978), "Review
of Don A. Dillman, Mail and Telephone Surveys,"
Journal of Business, 54 (4), 622-625.
Full Text
(PDF) - Review of the "Bible" for those involved in
survey research. This book was revised by Dillman in 2000 under the
title Mail and Internet Surveys.
J. Scott Armstrong and Terry S. Overton (1977), "Estimating Nonresponse Bias in
Mail Surveys," Journal of Marketing Research 14, 396-402.
Full
Text (PDF) - Bias occurs on questions that are
related to one's interest in a topic. One way to estimate nonresponse
bias is to look at trends over waves.
J. Scott Armstrong (1975), "Monetary
Incentives in Mail Surveys," Public Opinion Quarterly,
39, 111-116. Full
Text (PDF) - This meta-analysis showed that small
monetary incentives produced substantially larger response rates.
Robert W. Nason and J. Scott Armstrong (1972), "Role Conflict:
Society's Dilemma with Excellence in Marketing," Wharton Quarterly,
7 (1), 13-16.
Full Text (PDF)
David B. Montgomery and J. Scott Armstrong (1970), "Brand Trial
after a Credibility Change," Journal of Advertising Research,
10 (5), 26-32.
Full Text (PDF) - This study analyzed consumer
responses to Crest toothpaste after the tooth decay prevention class
was supported by the American Dental Association.
Return to top
J. Scott Armstrong (2004), Does an Academic Research
Paper Contain Useful Knowledge? No (p<.05) (Commentary on Mort,
et al., "Perceptions of Marketing Journals by Senior Academics in
Australia and New Zealand," pp. 51-61), Australian Marketing
Journal, 12 (2), 62-63.
Full Text
(PDF) - Assume that you were handed a paper randomly
selected from an academic journal in your field. What is the
likelihood that it contains useful knowledge? Our estimates suggest
that it would be less than 3%.
J. Scott Armstrong and Ruth A. Pagell
(2003), "Reaping Benefits from Management Research: Lessons from the
Forecasting Principles Project," Interfaces, 33 (6),
89-111 - Provides data showing
that invited papers are twenty times more likely to be important than
are those accepted through the traditional peer review system. This
paper was followed by commentary and by this reply,
See
review by Raymond Hubbard in
International Journal of
Forecasting, 20 (2004), 740-741.
J. Scott Armstrong (2003), "Incentives for
Developing and Communicating Principles: A Reply " Interfaces
33 (8), 2003, 109-111.
Full Text (PDF)
J. Scott Armstrong, Robert J. Brodie, and Andrew G. Parsons (2001),
"Hypotheses in Marketing Science: Literature Review and Publication
Audit," Marketing Letters, 12 (2),171-187.
Full Text (PDF) - Our survey showed that while experts
believe that scientific progress is aided by the method of multiple
hypotheses, this method is not widely used. Nor, as assessed by our audit
of the primary marketing journals, do many papers assess conditions.
J. Scott Armstrong (1998), "Management Science: What Does It Have to Do
with Management or Science?" Marketing Bulletin, 9 (May),
1-15. Full
Text (PDF) - Management scientists typically violate
accepted scientific procedures.
J. Scott Armstrong (1998), "Democracy Does Not
Make Good Science: On Reforming Review Procedures for Management
Science Journals, Interfaces, 28 (4), 88-91 - A Reply to Hugh
J. Miser (1998), "Reaction to Armstrong's 'Management Folklore and
management Science,'" Interfaces, 28 (4) 81-93.
Full Text
(PDF) - Reviewers should not be asked to recommend
whether or not a paper should be published.
J. Scott Armstrong (1997), "Peer
Review for Journals: Evidence on Quality Control, Fairness, and
Innovation," Science and Engineering Ethics, 3, 63-84. Full
Text (PDF) - A review of research from 68 empirical
studies on peer review, of which 12 were experimental studies.
Journal reviewing procedures are ineffective at selecting the most
useful papers, they are biased against new and important findings,
and they miss many errors.
J. Scott Armstrong (1996), "Management Folklore and Management
Science - On Portfolio Planning, Escalation Bias, and Such," Interfaces,
26, No. 4, 28-42. Full
Text (PDF) - Studies that challenge current
management practices are more likely to be rejected for journal
publication and less likely to be cited.
J. Scott Armstrong (1995), "Publication of Research on
Controversial Topics: The Early Acceptance Procedure," International
Journal of Forecasting, 11, Notes: 1-4.
Full
Text (PDF)
J. Scott Armstrong (1995), "Quality Control Versus Innovation in
Research on Marketing," Journal of Marketing Management, 11,
655-660.
Full Text (PDF) - An emphasis on quality control
leads to the rejection of useful papers.
Raymond Hubbard and J. Scott Armstrong (1994), "Replication
and Extensions in Marketing Rarely Published But Quite
Contrary," International Journal of Research in Marketing,
11, 233-248. Full
Text (PDF) Replications and extensions are rare
in marketing as they make up only one percent of the space. In
addition, 60% of the replications conflict in some way with the
original findings.
J. Scott Armstrong (1992), "Editorial
Policies for the Publication of Controversial Findings," International
Journal of Forecasting, 8, 543-544.
Full
Text (PDF) - Concluded, based on prior empirical
research, that reviewers should not be asked to make judgments on
whether a paper should be accepted for publication.
J. Scott Armstrong (1992), Review of T. Hancock,
J. Lane, R. Ray, and D. Glennon 1992, "The Ombudsman: Factors
Influencing Academic Research Productivity: A Survey of Management
Scientists," Interfaces, 22 (5), 26-38.
Full Text
(PDF) - Introduces a study and commentary on the
productivity of academics in management science. Prolific publisher
spent 1/3 less time on teaching activities - with no apparent effect
on teacher ratings.
J. Scott Armstrong and Raymond Hubbard (1991), "Does the Need for Agreement among Reviewers
Inhibit the Publication of Controversial Findings?" Behavioral and
Brain Sciences, 14, 136-137. Full
Text (PDF) - Improvements in the reliability of peer
review would pose additional barriers for research that produces important new
findings.
J. Scott Armstrong (1989), "Readability and Prestige in Scientific
Journals," Journal of Information Science, 15, 123-124.
Full
Text (PDF) - A follow-up to Armstrong 1980 paper on
"Unintelligible Management Research ..."
J. Scott Armstrong, Estella Bee Dagum, Robert Fildes, and Spyros
Makridakis (1986), "Publishing Standards for Research in Forecasting (An
Editorial)," International Journal of Forecasting, 2, 133-137.
Full Text
(PDF)
J. Scott Armstrong (1983), "The Importance of Objectivity and
Falsification in Management Science," Journal of Management,
9, 213-216. Full
Text (PDF) - A follow-up to Armstrong 1980 paper on
"Advocacy as a Scientific Strategy."
J. Scott Armstrong (1983), "Cheating in Management Science," with
comments by M. K. Starr and M. J. Mahoney, Interfaces 13 (4),
20-29. Full Text (PDF) -
Cheating seems to be rare in management science. When
it does occur, it is discussed widely, but informally.
J. Scott Armstrong (1982), "Barriers to Scientific Contributions: The
Author's Formula," Behavioral and Brain Sciences, 5 (June),
197-199. Full
Text (PDF) - This paper draws upon empirical
studies to produce guidelines on how to publish a paper in an academic
journal (e.g., do not study an important problem and do
not obtain surprising results).
J. Scott Armstrong (1982), "Is Review by Peers as Fair as It
Appears?" Interfaces, 12, 62-74, with comments by Martin
Starr, Herbert F. Ayres, Stephen M. Robinson, and Seth Bonder. Full
Text (PDF) - I conducted a survey of management
scientists to see if they could predict the outcome of a study that
resubmitted previously published papers to the journals that had
published them (but with new titles and authors). My sample was
unable to predict the outcome. This was the inaugural paper for the
"Ombudsman" column. I stand by all my proposals for paper review
except for blind receiving. Research since this paper has shown that
blind reviewing (shielding the reviewer) is detrimental.
J. Scott Armstrong (1982), "Research on Scientific Journals:
Implications for Editors and Authors," Journal of Forecasting,
1, 83-104.
Full Text (PDF) - For a more recent review of the
empirical evidence on peer review, see the Armstrong 1997 paper from
Science and Engineering Ethics.
J. Scott Armstrong (1980), "Advocacy as a Scientific Strategy:
The Mitroff Myth," Academy of Management Review, 5,
509-511. Full Text
(PDF) - Famous scientists often rely on advocacy to
advance their theories.
J. Scott Armstrong (1980), "Unintelligible Management Research and
Academic Prestige," Interfaces, 10, 2 (April), 80-86.
According to a study by U.G. Gupta in Interfaces (1997), 27,
No. 2, 85-101), this is the second most frequently cited paper in Interfaces
from 1970-1992. Full
Text (PDF) - Academics who have nothing to say
can add to their prestige by writing in a less intelligible manner.
J. Scott Armstrong (1979), "Advocacy and Objectivity in
Science," Management Science, 25 (5), 423-428. Full
Text (PDF) - An audit of papers
published in Management Science shows that most papers rely heavily on advocacy.
The method of multiple hypotheses was rarely used.
J. Scott Armstrong (1974), "Eclectic Research and Construct
Evaluation," in Models of Buyer Behavior: Conceptual, Quantitative,
and Empirical, Jagdish N. Sheth (ed.), New York: Harper and Row,
pp. 3-14.
Full Text (PDF) - Argues for splitting a research
budget into small projects with much different approaches to the
same problem.
Return to top
J. Scott Armstrong, "The Panalba Role Playing Case,"
adapted from an AMA proceedings paper, updated Nov. 2001. Full
Text (PDF) - A
learning exercise
J. Scott Armstrong (1990), "Review of Paul Blumberg, The Predatory Society: Deception in the American Marketplace"
(New York:
Oxford University Press, 1989), Journal of Marketing (October), 127-128. Full
Text (PDF) - The quasi-contract is written as part of
the product guarantee. in the event of any harm due to the use of
the product, people are then given an option to collect on the
policy - or to sue. Guess what's happening?
J. Scott Armstrong (1988), "Review of Peter W.
Huber, Liability: The Legal Revolution and Its Consequences." New
York: Basic Books, Inc. Full
Text (PDF) The quasi-contract was proposed as a way
to reduce product liability costs.
J. Scott Armstrong (1980), "Case of the Detrimental Drug:
Implications for the Stakeholder Theory of Directorship," Directors
and Boards, 4 (4), 22-39.
Full Text (PDF)
J. Scott Armstrong (1977), "Social Irresponsibility in Management,"
Journal of Business Research, 5 (September), 185-213. Full
Text (PDF) Subjects who role-played as members of
Upjohn's Board of Directors were unwilling to remove a detrimental
drug. However, when the boards were composed of stakeholders and when
social accounting was used, they did remove the drug.
J. Scott Armstrong (1975), "Review of Stanley Milgram,
Obedience To Authority (New York: Harper & Row, Publishers,
1974)," Journal of
Marketing 39 (July), 125. Full
Text (PDF)
Return to top
Richard H. Franke, J. Scott Armstrong, and
Paul M. Vaclavik (1998), "Competition vs. Profitability," presented
at the Strategic Management Society 18th Annual International
Conference -
Full Text (PDF) - Alternative financial criteria
and extended times found additional support showing that a
competitor-orientation harmed profits.
J. Scott Armstrong (1997), "Why Can't a Game Be More Like a Business? A
Review of Co-opetition by Brandenburger and Nalebuff," Journal of
Marketing, 61 (April), 92-95. Full Text
(PDF) This book offers interesting and creative ideas
about the use of game theory in business applications. Unfortunately,
it contains no evidence.
J. Scott Armstrong and Fred Collopy (1996), "Competitor Orientation:
Effects of Objectives and Information on Managerial Decisions and
Profitability," Journal of Marketing Research, 33, 188-199. Abstract,
Full Text
(PDF) When market share is an objective, profits
suffer, according to a series of laboratory studies and a long-term field study.
J. Scott Armstrong (1996), "Market Share Superstition"
(Letter), Sloan Management Review, Fall, 38, 4-5. Full
Text (PDF) Discusses another field
study concluding that market share should not be used as an objective.
J. Scott Armstrong (1991), "Strategic Planning Improves
Manufacturing Performance," Long Range Planning, 24 (4),
127-129. Full
Text (PDF) - Formal planning was superior in 20
studies and harmful in only 3 (with 5 ties). For manufacturing firms,
the score was 9 to 0.
J. Scott Armstrong (1988), "Review of Noel Capon, John G. Farley, and James M. Hulbert, Corporate
Strategic Planning (New York: Columbia University Press)," Journal of Marketing,
54 (1990).
Full
Text (PDF) In practice, formal planning was found to
be useful.
J. Scott Armstrong (1989), "On the Effectiveness of Marketing
Planning," New Zealand Journal of Business, 11 (October),
11-12.
Full Text (PDF)
J. Scott Armstrong (1988), "Review of Alfie Kohn, No Contest: The Case Against Competition," Journal of Marketing,
July. Full
Text (PDF) This book presents an impressive array
of evidence to suggest that a competitor orientation harms
performance under most conditions.
J. Scott Armstrong (1986), "The Value of Formal Planning for
Strategic Decisions: Reply," Strategic Management Journal, 7,
183-185.
Full
Text (PDF) This paper presents evidence to refute
a challenge to my conclusion that formal planning improves the
performance of firms. In effect, researcher bias does not explain the
results.
J. Scott Armstrong and David J. Reibstein (1985), "Evidence on the
Value of Strategic Planning in Marketing: How Much Planning Should a
Marketing Planner Plan?", from Strategic Marketing and
Management, (H. Thomas and D. Gardner, eds.), John Wiley &
Sons, Ltd. Full
Text (PDF) The intent here was to do a
meta-analysis on the value of strategic planning in marketing. No
evidence was found. This paper describes what evidence is needed.
J. Scott Armstrong (1983), "Strategic
Planning and Forecasting Fundamentals," in Kenneth Albert
(ed.), The Strategic Management Handbook, New York: McGraw Hill.
Full Text
(PDF)
This paper provides a
step-by-step planning procedure, with evidence on why it works.
J. Scott Armstrong (1982), "The Value of Formal Planning for
Strategic Decisions: Review of Empirical Research," Strategic
Management Journal, 3, 197-211. Full
Text (PDF) In this meta-analysis, formal
planning was superior in 10 of 15 comparisons from 12 studies.
Informal planning was more successful in only 2 studies.
Return to top of this
page
Return to Professor Armstrong's top page
Comments or questions
Copyright © 1994-2002 Trustees of the University of Pennsylvania
Certifying Authority: The Wharton School Marketing Department
(Count started 12/15/00)
|
|
