Declaration of Dr. Adam Werner in Response to Defedants’ Daubert Motion (Document 582-4)
Summary
Exhibit 3 to Document 582 in In Re: January 2021 Short Squeeze Trading Litigation, Case No. 21-2989-MDL-ALTONAGA/Damian (Case 1:21-md-02989-CMA, Document 582-4), entered on the FLSD docket June 21, 2023: the 28-page "Declaration of Dr. Adam Werner in Response to Defendants' Daubert Motion" dated June 21, 2023. Werner, asked by The Rosen Law Firm to answer the Robinhood defendants' June 7, 2023 motion to exclude Werner's opinions on market efficiency, responds that a one-year examination period before the Class Period is a generally accepted methodology; that removing price-movement and duplicate news articles does not change the collective test results; that the charge of a results-driven test design is speculation; and that the attack on the binomial test misreads its purpose. It also addresses the news-day selections of Mr. Fischel and Dr. Grenadier.
Summary drafted by a model from the document's text below and checked by script against that text before publication. It is a navigation aid, not a reading of what the document proves. Where AI is used
- Date
- 2023-06-21
Cite this document
“Declaration of Dr. Adam Werner in Response to Defedants’ Daubert Motion (Document 582-4).”
Original at archive.org, dated 2023-06-21. Archived at Pandemic Darlings, https://pandemicdarlings.com/source-documents/2023-06-21-covid-era-misc/
, SHA-256 4ada1715d61e7320dafe137591aadbd431422fe0e12785c015b5100c8a0c3987.Full text
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 1 of 28
EXHIBIT 3
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 2 of 28
UNITED STATES DISTRICT COURT
SOUTHERN DISTRICT OF FLORIDA
In Re: JANUARY 2021 SHORT SQUEEZE
TRADING LITIGATION Case No. 21-2989-MDL-
ALTONAGA/Damian
This Document Relates to: All Actions Concerning the Federal Securities Laws
DECLARATION OF DR. ADAM WERNER
IN RESPONSE TO DEFEDANTS’ DAUBERT MOTION
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 3 of 28
TABLE OF CONTENTS
I. INTRODUCTION ...............................................................................................................1
II. SUMMARY OF RESPONSE TO DAUBERT MOTION...................................................2
III. RESPONSES TO DEFENDANTS’ DAUBERT MOTION................................................2
A. Examining the One-Year Period Prior to a Class Period Alleged to Have
Been Affected by Stock Price Manipulation is Appropriate and Supported
by the Literature .......................................................................................................3
1. Using a Dummy Variable to Control for Potentially Abnormal
Price Movements on January 27th Is Methodologically Sound ...................5
2. Defendants’ Suggestion that I “Commingled” Two Different Data
Sets Is Not Well Taken ................................................................................6
B. The Collective Tests I Performed Are Methodologically Sound and
Robust to Defendants’ Criticisms ............................................................................8
1. My Collective Test Event Selection Methodology is Informed by
Various Factors, Not Only Halliburton II ....................................................9
2. Removing The “Noise” From my Collective Tests Have No Impact
on My Conclusions ....................................................................................14
3. Defendants’ Attack on My Data’s Inclusion of News Articles
Discussing Only Price Movements is Hypocritical and
Undermined by Robinhood’s Own Statements..........................................15
C. Defendants’ Claim that my Collective Test Design was Results-Driven is
Pure Speculation ....................................................................................................16
D. Defendants’ Critiques of the Binomial Test I Performed Ignores My
Testimony and the Testimony of Mr. Fischel ........................................................21
2
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 4 of 28
I, Adam Werner, declare and state, under penalty of perjury, that the following is true and
correct to the best of my knowledge, information, and belief. If called to testify, I could and would
testify competently to the following facts.
I. INTRODUCTION
On June 7, 2023, I received Defendants Robinhood Markets, Inc., Robinhood
Financial LLC and Robinhood Securities, LLC’s Memorandum of Points and Authorities in
Support of their Motion to Exclude the Opinions of Dr. Adam Werner on Market Efficiency
(“Defendants’ Daubert Motion”). I am now asked by The Rosen Law Firm to consider and
evaluate the arguments and conclusions in Defendants’ Daubert Motion and provide a response.
I make this declaration in support Plaintiffs’ opposition to Defendants’ Daubert
Motion.
1
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 5 of 28
II. SUMMARY OF RESPONSE TO DAUBERT MOTION
RESPONSE 1: An examination period spanning one-year of data prior to the
Class Period is a widely used and generally accepted methodology. Focusing
the examination period on the year prior to the Class Period is justifiable
given the allegations of stock price manipulation during the Class Period.
RESPONSE 2: Defendants’ claims that my collective test’s use of a broad
(but objective) selection methodology to identify all “news days”, including
news of price movements, may have resulted in noise being introduced into
my “news days” sample misunderstands the design of my test. Moreover, the
results of the test do not change when I remove the news articles that were
discussing price movements, discussing topics arguably unrelated to the
company, and those articles that were substantially similar in substance (I.e.
duplicates).
RESPONSE 3: Defendants’ claims that I deviated from my past collective
test methodologies in order to arrive at favorable results is pure speculation.
As I explained in my deposition, my collective test methodology was designed
to eliminate the inherent subjectivity within the selection criteria suggested
by Defendants. Further, I have used this exact collective test methodology in
subsequent cases.
RESPONSE 4: Defendants’ attacks on the Binomial test I performed in the
Werner Rebuttal fundamentally misunderstands the intent and context of
my analysis. The purpose of the binomial test was to address their experts’
criticism that I evaluated Cammer factors in the wrong time period and
should have evaluated the January 2021 pre-class period rather than the one-
year period. I performed the binomial test using the news days selected by
both Mr. Fischel and Dr. Grenadier to evaluate whether the Affected Stocks’
prices moved in response to news in the immediate pre-class period.
Whether the prices move without news is not the pertinent question.
Moreover, their selection of non-news days is biased, unreliable and not
subject to reliable testing.
III. RESPONSES TO DEFENDANTS’ DAUBERT MOTION
As an initial matter, in addition to the Cammer factors 1-4 and Krogman tests
performed in the Werner Rebuttal Report with respect to the immediate pre-class period (January
4 - 27, 2021), the Cammer factor 5 news/no news test I conducted included the “news” days
identified by both Mr. Fischel and Dr. Grenadier. Defendants’ Daubert motion, however, appears
to be a sur-rebuttal, presenting a new statistical analysis challenging my work, but based only on
2
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 6 of 28
the work of Dr. Grenadier. Specifically, Defendants’ present a new binomial test as a response to
the binomial test I performed in my rebuttal report. However, the data and identification of “news
days” in Defendants’ new binomial test uses only the news days identified by Dr. Grenadier and
excludes Mr. Fischel’s identification of “news days” entirely.
Robinhood’s decision to abandon Mr. Fischel’s news days is a tacit admission that
his analysis is doubtful and underscores the subjectivity and high potential error rate in the
subjective method Mr. Fischel (and, by extension, Dr. Grenadier) used to select news days.
A. Examining the One-Year Period Prior to a Class Period Alleged to Have Been
Affected by Stock Price Manipulation is Appropriate and Supported by the
Literature
Defendants take issue with the fact that I employed a one-year examination period
prior to the Class Period to study market efficiency.1 Specifically, Defendants argue that such an
examination period does not answer the question of whether the Seven Affected Companies
traded in an efficient market during the Class Period itself. However, I was never asked to assess
market efficiency during the Class Period. As I explained in the first paragraph of my opening
report:
“I was asked by The Rosen Law Firm, counsel for the Plaintiffs, to determine
whether the common stock of AMC Entertainment Holdings, Inc. (‘AMC’), Bed
Bath & Beyond Inc. (‘BBBY’), BlackBerry Limited (‘BB’), Express, Inc.
(‘EXPR’), GameStop Corp. (‘GME’), Koss Corporation (‘KOSS’), Tootsie Roll
Industries, Inc. (‘TR’), and the American Depositary Shares of foreign-issuers
Nokia Corporation (“NOK”) and trivago N.V. (‘TRVG’) (collectively the
‘Affected Companies’ or ‘Affected Stocks’) traded in efficient markets prior to the
Class Period.”2
1
Defendants’ Daubert Motion, ¶8.
2
Werner Declaration, dated February 16, 2023, ¶1.
3
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 7 of 28
In this case, Plaintiffs allege that, during the Class Period, Robinhood imposed
trading restrictions that prevented investors from participating in the marketplace and
manipulated the price of the Affected Companies’ stocks. Such direct manipulation would
reasonably alter the stock price dynamics and market efficiency of any company affected by the
restrictions. Thus, using an examination period prior to the Class Period is methodologically
sound.
In the Werner Rebuttal Report, I provided a similar explanation for why a study of
market efficiency using data during the Class Period would be inappropriate:
Furthermore, because Plaintiffs allege that Robinhood manipulated the market for
the Affected Companies by imposing trading restrictions and impediments
beginning on January 28, 2021, I do not examine the period from January 28, 2021
through February 4, 2021 for market efficiency. Market manipulation and enforced
trading impediments would very likely affect the efficiency of said markets. It
would be unsurprising if the efficiency of the market for the nine Affected
Companies waned following Robinhood’s interference; inclusion of this period
would therefore bias any test against a finding of market efficiency.3
Dr. Grenadier similarly testified that a market is not efficient when it is subject to
manipulation. 4 Thus, it would be pointless to test efficiency during the period of alleged
manipulation when Robinhood’s restrictions were in place.
The choice of studying a one-year period prior to the Class Period given the
allegations that the markets for the Affected Stocks were manipulated during the Class Period, is
methodologically sound.
3
Werner Rebuttal Report, ¶17.
4
See, e.g., Grenadier Deposition, at 41:20 – 42:25, “I would view a manipulation as something that would make a
market that is efficient not efficient.”
4
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 8 of 28
1. Using a Dummy Variable to Control for Potentially Abnormal Price
Movements on January 27th Is Methodologically Sound
In their motion, Defendants attack my exclusion of January 27, 2021 from my
regression analysis as being logically inconsistent with the fact that January 27, 2021 is part of
my efficiency opinion. 5 Defendants’ critique fundamentally misunderstands dummy variables
and conflates the regression analysis with tests of market efficiency.
As I explained in the Werner Declaration, using dummy variables to control for
potentially atypical observations in the regression estimation period so that the model parameters
can better reflect typical stock price dynamics, is a widely used and generally accepted
methodology.6 Because the purpose of a regression model is to determine what typical stock price
behavior looks like, dummy variables are used to remove days that are a priori known to be
abnormal so that the regression model can properly capture this typical price behavior. In the case
of January 27, 2021, I was aware after reading the Complaint that this is a date alleged to be
highly impactful on the prices of the Affected Companies and Robinhood’s subsequent actions.
Thus, by definition, this date could not be considered typical and would not warrant inclusion in
a regression model estimating typical stock price behavior. Removing January 27, 2021, via the
use of a dummy variable is therefore consistent with the aim of the regression analysis.
A regression analysis, however, is not the same thing as a test of market efficiency.
After establishing a benchmark of typical stock price movements, the regression analysis then
estimates the “abnormal”, or residual, return for every day, which includes January 27, 2021, in
5
Defendants’ Daubert Motion, pp.7-8.
6
See, e.g., “Event Studies with a Contaminated Estimation Period,” by Nihat Aktas et al., Journal of Corporate
Finance, vol. 13., 2007; “Testing for Market Efficiency: A Comparison of the Cumulative Average Residual
Methodology and Intervention Analysis,” by David Larcker et al., Journal of Financial and Quantitative Analysis,
vol. 15, no. 2, 1980.
5
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 9 of 28
the instant case, during a specified period of study (in this case, the one-year period prior to the
Class Period). These abnormal returns are then used as inputs into either event studies or
collective event studies like the one I performed in order to draw inferences about market
efficiency. That is, the collective tests I performed are a separate analysis from the regression
analysis.
Defendants’ critique that somehow my use of a dummy variable on January 27,
2021, from my regression model means I must have also excluded this day from my collective
event study analysis conflates the two distinct analytical steps of regression analysis and market
efficiency analysis as a singular exercise. In fact, I did include January 27, 2021, in my analysis
of market efficiency.7 Defendants’ claims to the contrary are baseless.
2. Defendants’ Suggestion that I “Commingled” Two Different Data Sets
Is Not Well Taken
Defendants claim that my purported “commingling” of data in my use of the one-
year estimation period prior to the Class Period is “improper” because it “commingles a general
data set with a narrower one.”8 However, Defendants offer zero evidence other than anecdotes
and speculation that there is even a distinction between this so-called “general data set” and the
“narrower data set”. It is unscientific to assume that there is “substantial variation” in data
properties for the Seven Affected Companies based only on observational comparisons of
historical stock price movements over the past 25 years (the analysis Dr. Grenadier performed in
his opening report that Defendants rely on here).9 Simply noting that price movements are larger
7
Tangentially, my conclusions with regard to market efficiency are also robust to excluding January 27, 2021 from
my collective test analysis.
8
Defendants’ Daubert Motion, p. 9.
9
Defendants’ Daubert Motion, p. 9.
6
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 10 of
28
during January 2021 than historical price movements does not go far enough. While price
movements may appear to be extreme when compared to 25 years of historical data, there are
more precise statistical tests that can be applied (and I apply them in this report below) to assess
whether the price dynamics for the Seven Affected Companies during the one-year period I
studied was truly different enough to warrant special treatment. Moreover, the very fact that Dr.
Grenadier and Mr. Fischel disagree among themselves as to the pre-Class period that should be
studied suggests that both Defendants’ experts are drawing artificial lines when deciding where
the supposedly more-narrow data set actually begins.
Rather than assume that there must be “substantial variation” in data properties
based on observed differences between unrelated companies in completely different economic
environments, whether there actually is a difference can be tested directly in a scientific manner.
One such test is the Breusch-Pagan test for heteroskedasticity. Heteroskedasticity means that the
residual price movements of a particular stock are not normally distributed; this is akin to saying
that the existence of heteroskedasticity means that one can assume there was in fact “substantial
variation” in the underlying data properties. The opposite of heteroskedasticity, or
homoskedasticity, means that the residual price movements are normally distributed – i.e. there
is no “substantial variation” in the underlying data properties. The academic literature discusses
the implications of heteroskedasticity as well as common tests used to assess whether it is present:
“The least squares model assumes that the disturbances have uniform variance,
known as homoscedasticity. With homoscedastic disturbances, all values of Y
corresponding to various values of X have equal importance. But what if the
variance of the disturbance relates to the size of the independent variable? … With
heteroscedasticity, least squares estimates of the coefficients remain unbiased but
the standard errors are biased, thereby leading to incorrect statistical inferences.
One simple test to detect heteroscedasticity is visual inspection. For instance, the
smaller residuals to the right of the graph in Exhibit 9-3 indicate a relation between
the error variance and the independent variable. More formal tests to detect
7
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 11 of
28
8
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 12 of
28
my “news” sub-sample including articles that Defendants posit would have no “valuation-
relevance” the Affected Companies I studied, the conclusions of my analysis are unreliable.
Defendants are wrong on both points.
1. My Collective Test Event Selection Methodology is Informed by Various
Factors, Not Only Halliburton II
While defendants claim that “Dr. Werner’s erroneous understanding of Halliburton
II led him to conduct a news/no news test that includes ‘news’ unrelated to the economic outlook
of the firm,” this is not true.11 As I testified in my deposition, in addition to my reconsideration
of Halliburton II, my test design was informed by my work in previous cases as well as the
critiques I received in those past cases.12
In applying my typical collective test designs, i.e. “news” being defined as either
earnings announcements, Form 8-K/Form 6-K filings, or Company press releases, it has been my
experience that none of these selection criteria generates a fully objective set of material news
days. In the case of earnings, it is unreasonable to expect every earnings announcement to
generate a statistically significant price reaction as there exists a multitude of valid (and consistent
with market efficiency) reasons for why earnings announcements may have no impact on a
security’s price. For example, earnings that were expected, were unsurprising, earnings for a
biotechnology company that has little or no revenue, or earnings reports that contain
countervailing information within earnings releases that have a net-neutral effect should result in
statistically insignificant stock price reactions. Such observations could lead one to conclude that
11
Defendants’ Daubert Motion, p. 12.
12
See, e.g., Werner Deposition in In Re NIO, Inc. Securities Litigation, Eastern District of New York, Case No. 1:19-
cv-01424-NGG-JRC and Reply Declaration of Dr. Adam Werner, dated August 6, 2018, in Ivan Nibur, et al., v.
Sandridge Mississippian Trusts, et al., Western District of Oklahoma, No. 15-cv-00634-SLP.
9
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 13 of
28
the companies associated with the aforementioned earnings were in fact inefficient which is not
necessarily the case. These phenomena have been studied and observed in the academic literature:
“Prices can react efficiently to information even though the price reactions
themselves are not so unusual in size as to approach statistical significance. The
question for efficiency is whether the price reaction accurately reflected the impact
of the news on the value of the firm—whether that impact was .1, .5, 1, 1.96, 3, or
more standard deviations of value change. Thus, while a statistically significant
reaction to a firm-specific news event is evidence that information was reflected in
the price (absent confounding effects), the converse is not true—the failure of the
price to react so extremely as to be two standard deviations from average does not
establish that the market is inefficient; it may mean only that the correctly-sized
value impact that occurred was less than 1.96 standard deviations from the mean.
While some courts have been sensitive to this distinction by rejecting bad
arguments to the contrary, other courts have remained inattentive to this fact, which
has generated inaccurate findings in some securities cases.”13
Furthermore, a selection criteria containing only earnings announcements is often
too restrictive to capture all “news days”. For example, in the In Re Alibaba Group Holding
Limited Securities Litigation matter, Dr. David Tabak described a selection criterion using only
earnings announcements as “overly restrictive” and noted, by using such a criterion, the set of
“non-news days” would be polluted by “news days”.
“To remove any possible subjectivity on my part as to which ‘news days’ were
selected for examination, I performed one analysis by defining news days as days
with Alibaba earnings announcements. This is, of course, an overly restrictive
definition of ‘news days,’ and results in comparing (1) a set of particular news days
(though in this case, just a single day) with (2) a mixed set composed of both non-
news days and days that would (but for the restrictive definition) be considered
news days.”14
13
“Event Studies in Securities Litigation: Low Power, Confounding Effects, and Bias,” by Alon Brav and J. B. Heaton,
Washington University Law Review, Vol. 93, No. 2, 2015, p. 602.
14
Expert Report of David I. Tabak, Ph.D., In Re Alibaba Group Holding Limited Securities Litigation, dated March
12, 2018, ¶39.
10
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 14 of
28
In this case, using this particular definition of news, led Dr. Grenadier to implicitly
conclude that Blackberry, Bed Bath & Beyond and Nokia showed indications of trading in
efficient markets prior to the Class Period.15
Form 8-Ks/Form 6-Ks as well as company press release suffer from the flaw of
being news that is curated by the company itself. These forms of disclosure often contain the
exact type of non-valuation relevant information that Defendants criticize my test design for
having included. For example, on July 28, 2020, Bed Bath & Beyond filed a Form-8K disclosing
the following:
“On July 28, 2020, Bed Bath & Beyond Inc. (the “Company”) issued a press release
announcing that Lynda Markoe has been appointed as Executive Vice President,
Chief People & Culture Officer. Ms. Markoe will join the Company in September
2020. A copy of the press release is attached hereto as Exhibit 99.1 and is
incorporated by reference into this Item 7.01. The information in Item 7.01 of this
Current Report on Form 8-K is being furnished and shall not be deemed “filed” for
the purposes of Section 18 of the Securities Exchange Act of 1934, as amended (the
“Exchange Act”), or otherwise subject to the liability of such section or
incorporated by reference in any filing under the Securities Act of 1933, as
amended, or the Exchange Act, except as shall be expressly set forth by specific
reference in such a filing.”16
It is unclear how a disclosure about the appointment of a new “People & Culture
Officer” could be considered “valuation-relevant” information that would likely cause a
statistically significant price reaction.
15
Grenadier Report, dated March 28, 2023, Exhibit-1A.
16
Bed Bath & Beyond, Form 8-K, filed July 28, 2020.
11
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 15 of
28
Using this particular definition of news, Form 8-K/Form 6-K filings, Dr. Grenadier
now finds that AMC and Bed Bath & Beyond showed indications of trading in efficient markets
prior to the Class Period.17
Company press releases are not a better alternative in terms of their ability to
identify “valuation-relevant” news. For example, on March 6, 2020, AMC issued a press release
titled “’Clueless’ is 25! Would Moviegoers Miss a Big-Screen Party for the Era-Defining
Comedy? As If!” The first paragraph of this article, sourced from PR Newswire, contains the
following:
“Cinema Audiences Will Be Totally Buggin’ When Cher and Her Friends From
Beverly Hills Return in Writer-Director Amy Heckerling’s Classic DENVER,
March 6, 2020 /PRNewswire/ -- It’ll be the phattest high school reunion ever when
Cher Horowitz and the Class of 1995 come back to the big screen as Fathom Events
and Paramount Pictures celebrate the 25th anniversary of writer-director Amy
Heckerling’s film ‘Clueless’ for three days only this May.”18
While this is an AMC press release, it is unclear how Defendants would consider
this to be “valuation-relevant” information. If anything, applying Defendants’ selection criteria
would place unwarranted emphasis on the expectation that news such as this press release should
affect stock price movements.
Interestingly, under the definition of company press releases as news, Dr. Grenadier
again found that Bed Bath & Beyond showed indications of trading in an efficient market prior
to the Class Period.19 That Dr. Grenadier’s application of these three different selection criteria
17
Grenadier Report, dated March 28, 2023, Exhibit-1B.
18
Content Analysis: An Introduction to Its Methodology, by Klaus Krippendorff, 2nd edition, 2004, p. 59 (emphasis
in original).
19
Grenadier Report, dated March 28, 2023, Exhibit-1C.
12
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 16 of
28
(earnings announcements, Form 8-K/Form 6-K, and Company press release) yield different
results and implications for market efficiency indicates that such selection methodologies are not
robust in their ability to identify efficiently traded stocks. By not limiting the definition of “news”
to information curated by the companies at issues in the case, a more objective selection
methodology is possible.
Applying a broad, but objective, news selection criteria is exactly the methodology
I employed in the Werner Declaration. In Appendix-B of that Declaration, I explained that my
selection criteria began by searching for all news articles published about each of the Affected
Companies that were available from Factiva. Other than specifying my search parameters for each
Affected Company, the one-year pre-Class Period that I examined, and removing identically
duplicate articles, no other adjustment was made to the sample of news articles that Factiva
provided. For every Affected Company, these news articles included (but were not limited to)
earnings announcements, company press releases, as well as Form 8-Ks/Form 6-Ks. Thus, while
I did not focus my event selection on the too-narrow definition news suggested by Defendants,
my event selection nonetheless captured all events that Defendants deemed to be newsworthy.
Rather than place the responsibility of event selection onto the companies at issue
in the case, I instead looked to all market participants’ disseminations of information in order to
capture a set of days that the market itself deemed important enough to warrant a high frequency
of news dissemination. This concept is not novel. As I explained my Declaration, the idea that the
frequency of news releases on a particular topic is an indicator of its import has been established
in content analysis studies:20
20
Werner Declaration, ¶68.
13
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 17 of
28
“The frequency with which a symbol, idea, reference or topic occurs in a stream of
messages is taken to indicate the importance, of attention to, or emphasis on that
symbol, idea, reference, or topic in the messages.”72
While using this very broad definition of news captures all news regardless of
whether it is “valuation relevant”, it eliminates any subjectivity. If a statistical test comparing
the two samples then finds a difference in the frequency of statistical significance among the two
samples, the conclusion is that a greater frequency of news coverage is associated with a greater
frequency of statistically significant price movements, thereby demonstrating a cause-and-effect
relationship between news flow and price movements. Thus, as I posited in my Declaration, in an
efficient market, one would expect days with a higher number of news releases to be more likely
to exhibit abnormal security price movements.21 I designed my collective event study tests to
assess this hypothesis and I determined that for the Seven Affected Companies, there was indeed
a statistically significant difference between atypical stock price movements and the frequency of
news coverage. I therefore concluded that the stocks of the Seven Affected Companies traded in
efficiency markets.
2. Removing The “Noise” From my Collective Tests Have No Impact on
My Conclusions
While defendants criticize the fact that my broad, yet objective, selection criteria
resulted in extraneous or improper articles being included as “news”, 22 when given the
opportunity to rebut my conclusions, their experts chose not to demonstrate that making such an
adjustment would affect any of my conclusions because in fact, such adjustments, do not
materially change my findings.
21
Werner Declaration, ¶69.
22
Defendants’ Daubert Motion, pp. 15-16.
14
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 18 of
28
In response to Defendants’ critique, I present the results of my collective test
analysis for the Seven Affected Companies if I were to remove articles with potentially unrelated
news, discussions of price movements only, and duplicate articles discussing the same
information, as a response to Defendants’ critique in Table-2 below.
Table-2: Collective Test Results for Relevant Period After Filtering for “Noise”
High Information Flow Days Low Information Flow Days
Fisher's Exact
Affected Statistically % of Significant Statistically % of Significant Test
[1]
Company Total Days Significant Days Days Total Days Significant Days Days P-Value
AMC 26 7 26.92% 227 12 5.29% 0.12%
BB 27 5 18.52% 226 7 3.10% 0.45%
BBBY 28 9 32.14% 225 12 5.33% 0.01%
EXPR 30 6 20.00% 223 8 3.59% 0.26%
GME 27 7 25.93% 226 6 2.65% 0.01%
NOK 26 6 23.08% 227 11 4.85% 0.37%
TRVG 25 4 16.00% 228 10 4.39% 3.78%
Sources: Factiva, CRSP, Bloomberg, and computations performed by Crowinshield Financial Research, Inc.
Note:
[1] Fisher's Exact Test p-values that are less than 5% are statistically significant at the 95% confidence level, or greater.
As shown in this table, none of the qualitative test results change for the Seven
Affected Companies. As such, Defendants’ arguments regarding my test methodology are without
merit and have no impact on my conclusions even after making the adjustments Defendants
contend are required but chose not to perform themselves.
3. Defendants’ Attack on My Data’s Inclusion of News Articles Discussing
Only Price Movements is Hypocritical and Undermined by Robinhood’s
Own Statements
Defendants criticize the quality of the sample of news articles I used in my
collective tests as “defective” because “it includes news that is literally only about the stock price
movements.”23 However, it would appear that Robinhood itself considers information about stock
23
Defendants’ Daubert Motion, p. 12.
15
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 19 of
28
price movements relevant enough to offer this information to its own customers as a way to “help
them stay informed”. According to Robinhood CEO, Vlad Tenev:
“Our business has come under attack by critics who insist that our platform is
‘gamified.’ They point to features such as lists of stocks and exchange-traded funds
that help people discover investments and notifications about stock movements that
help them stay informed. We designed these features, many of which are common
in our industry, to make it easier and more delightful for users to stay informed.
Investing isn’t a game, but must it be grim and difficult to understand?”24
It is hypocritical of Defendants to criticize my inclusion of price movement news
articles in my data sample when Robinhood itself participated in the dissemination of exactly
such information and characterized such information as making “it easier and more delightful for
users to stay informed.”25
C. Defendants’ Claim that my Collective Test Design was Results-Driven is Pure
Speculation
In their motion, Defendants claim that my analysis “failed to follow the same
methodology [I] followed in prior cases” and Defendants further speculate that “he did so because
applying his standard methodology would have led to the conclusion that most of the Affected
Stocks did not trade in efficient markets.”26 Both of these claims are false.
24
“Robinhood Users Come Under Attack,” by Vlad Tenev, The Wall Street Journal, September 27, 2021.
25
“Robinhood Users Come Under Attack,” by Vlad Tenev, The Wall Street Journal, September 27, 2021.
26
Defendants’ Daubert Motion, p. 16.
16
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 20 of
28
My collective event study analysis followed the same methodology that I have been
using to test for market efficiency for the past several years. As explained in the Werner
Declaration, and in numerous other cases in the past, my collective test methodology tests whether
a company’s stock responds to company-specific information by examining the incidence rate of
statistically significant returns on “news” dates and “non-news” dates. Plainly, the methodology
looks at every day during the relevant period and, using an objective criterion, defines each date
as a “news” day or a “non-news” day, where “news” days are defined as having a greater flow of
information. The test then determines how frequently the security price exhibits a statistically
significant response on “news” days and “non-news” days. Based on these frequencies, a
statistical test is conducted to determine whether there is a statistically significant difference
between the reaction frequency of “news” dates and “non-news” dates. If the test finds a
statistically significant difference, then that is evidence that the security responded to the flow of
information during the period of study, and therefore exhibited market efficiency.
I used this methodology in this case to test the securities of the Seven Affected
Companies, and have used this methodology in numerous cases over the past several years.27
Thus, Defendants’ claim that I “failed to follow the same methodology [I] followed in prior cases”
is patently false.
In the implementation of the methodology, an expert must decide the parameters of
the definition of “news” days. This decision will have some variation from case to case, as not all
factors that are relevant to one company will be the identical relevant factors to the next. Investors
27
See, e.g., In Re Horsehead Holding Corp. Securities Litigation, District of Delaware, Civil Action No. 16-cv-292-
LPS-CJB; In Re XL Fleet Corp. Securities Litigation, Southern District of New York, Case No. 1:21-cv-02002-JLR;
Ivan Nibur, et al., v. Sandridge Mississippian Trusts, et al., Western District of Oklahoma, No. 15-cv-00634-SLP; and
Peter Voulgaris v. Array Biopharma, Inc., District of Colorado, Civil Action No. 17-cv-02848-SRV.
17
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 21 of
28
may focus on different metrics for large, mature companies than they will for smaller, growing
companies. Companies in different industries will have different types of news that investors
consider important. The most important group of information events – and thus a good candidate
for “news” events in the collective test – for some companies may be earnings announcement
events, while for other companies, investors may weigh the import of other corporate events much
more heavily. For example, results of a pending drug trial may be the most important news events
for investors in a pharmaceutical company, while production estimates may be the most important
news events for an oil and gas producer investors. In prior cases, I have used these precise
selection criteria for my definition of “news” dates.28
Plainly, my collective event study analysis methodology is consistent between my
reports. The precise selection criteria of a “news” event can and should vary from case to case to
account for the at-issue company’s industry and characteristics. That there is no singular criteria
for important “news” days for all companies across all industries is underscored by Defendants
own analysis in this case. Defendants’ own experts – Mr. Fischel and Dr. Grenadier – disagreed
on what constituted news days over a five-day period.
In the instant case there were nine companies to study, not just one as in past reports,
of varying sizes, industries, and characteristics. The most important “news” days in any given
period for these companies would not be captured under a single narrow criteria. Thus, because
large, momentous events for public companies are often accompanied by greater news coverage,
the selection criteria for all companies were dates that had the greatest volume of news coverage.
Therefore, for each company, the “news” bucket would contain important information events,
28
Ivan Nibur, et al., v. Sandridge Mississippian Trusts, et al., Western District of Oklahoma, No. 15-cv-00634-SLP
and Peter Voulgaris v. Array Biopharma, Inc., District of Colorado, Civil Action No. 17-cv-02848-SRV.
18
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 22 of
28
even if those precise events varied from company to company. Using this definition of news
allowed me to apply a consistent definition of news across all nine Affected Stocks.
Defendants’ assertions that I selected this definition of news in a results-driven
manner because I knew the other definitions of news events would not yield results favorable to
market efficiency29 is patently false. Defendants point to two sections in my deposition testimony
to make the claim that when “Dr. Werner first embarked on his work in this case, he actually
began by using the approach he had in all of his prior cases.”30 This is not what I said in my
deposition. In the first section of my deposition that Defendants cite to, the citation Defendants
use leaves out roughly half of my full response. While Defendants chose to focus on the fact that
I applied a different event selection criteria than I had used in past cases, they completely ignore
my testimony where I explained exactly why I did so:
“Q. You can't point to a single report that you submitted before you submitted your
opening report in this matter where you applied this version of the news/no news
test; isn't that right, sir?
A. I believe that's correct. But, I mean, maybe you will understand it better if I
explain the evolution. So when we had our discussion earlier about earnings days,
right, and so it could be the case that, you know, each earnings day met expectations
and there was no stock price movement. So what I was doing in this case was trying
to find a consistent methodology that would work across all companies. So, you
know, earnings announcements are extremely restrictive. I have stopped using that
at least at this point. In terms of looking at 8-Ks or news releases, in the NIO case
I discovered that when I looked at the underlying information, there were
discussions of things like moving stock, like a CEO moving from his personal
account to a trust. So it became obvious to me that using 8-Ks or press releases
could be over or under-inclusive with regards to news that investors might find
material. And then based on court acceptance of these tests that I ran in this case as
29
Defendants’ Daubert Motion, pp. 16-17.
30
Defendants’ Daubert Motion, p. 17.
19
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 23 of
28
well -- as well as reports that I've seen by other experts, it seemed to me that this
was the most objective way to measure news.
And, by way of example, you know, you have two experts who arrive at different
conclusions with regards to which days contained material information over a five
or six-day period. So it seemed to me that I wanted to be as objective as possible,
and so I have adopted this current methodology on this report and every report
since.”31
Nowhere in my deposition testimony did I state that I ran other tests for market
efficiency prior to the one I presented in the Werner Declaration. As shown above, what I
explained in my deposition was that in past cases where I had applied the overly narrow
definitions of news recommended by Defendants, such applications were critiqued for being too
subjective in differentiating between news vs. non-news and incomplete in their ability to actually
capture material information. In my work on this case and subsequent cases, I took these previous
critiques into consideration and applied a methodology responsive to the criticisms I’ve
experienced.
Defendants also cite to pages 229:2 to 235:2 from my deposition in their argument
that my tests were results-driven.32 I do not disagree with Defendants’ representation that I did
read news articles about each of the Affected Companies prior to performing any analysis of
market efficiency. However, reading news on a company in order to better understand the industry
and markets that the Affected Companies were operating in is not the same as testing for market
efficiency as Defendants attempt to portray. Examining a company’s industry and business model
by reading articles written about that company is one of the first steps I take before employing
any event selection methodology. In order to determine what constitutes “news” for a particular
31
Werner Deposition, at 173:24 to 175:19.
32
Defendants’ Daubert Motion, p. 17.
20
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 24 of
28
company, one must first understand the company they are studying. This is the principle I applied
in determining my event selection methodology.
Furthermore, the selection criteria I applied is also consistent with the analysis that
I have provided in other cases. For example, in Bo Shen v. Exela Technologies, Inc.,33 I also
provided a collective event study analysis where the selection criterion was based on the volume
of news during the relevant period. Thus, not only is my collective event study methodology
consistent with my prior testimony, but the precise selection criteria to define “news” days within
that methodology is also consistent with my work in other cases. Defendants’ claim that my
analysis “failed to follow the same methodology [I] followed in prior cases” is baseless.
D. Defendants’ Critiques of the Binomial Test I Performed Ignores My
Testimony and the Testimony of Mr. Fischel
In the Werner Rebuttal Report, I explained that the periods of study examined by
both Mr. Fischel and Dr. Grenadier are extraordinarily short. Mr. Fischel studies a period of only
17 trading days in the immediate pre-class period while Dr. Grenadier studies a period of only 11
trading days, divided between the pre-class (5 days) and Class (6 days) periods. I also explained
in that report that, in statistical and economic analysis, it is generally advised for scientific studies
to examine at least 30 observations so that the size of the sample is large enough to lead to robust
conclusions.34 A low number of observations is more likely to exhibit extreme outcomes simply
due to random chance. Thus, as I explained in that previous report, I place relatively less weight
on the results of any study focusing on this short examination period due to the extraordinarily
low number of trading days each expert chose to examine.
33
Bo Shen v. Exela Technologies, Inc., Northern District of Texas, Case No. 3:20-cv-00691-D.See also, In Re XL
Fleet Corp. Securities Litigation, Case No. 1:21-cv-02002-JLR.
34
Reference Manual on Scientific Evidence, 3rd ed., Washington: The National Academies Press, 2011, p. 255.
21
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 25 of
28
I also explained in my Rebuttal Report that both Dr. Grenadier and Mr. Fischel
conducted what is typically known as a “backwards event study” in order to evaluate “value
relevant” news. In a backwards event study, there is no a priori event selection – in this case a
priori selection of news days to study. In other words, there are no specific events being identified
to be tested to determine whether a stock responds to new information about the company being
studied. Rather, in a backwards event study, the regression analysis is performed first, such that
the expert already knows which days during their study period are statistically significant (and
which days are not) prior to examining any actual news event to determine if it is material
valuation relevant news. The issue with this form of event study approach is that any event
examined after the fact is biased by the expert’s prior knowledge of the statistical results of their
test, rendering the event study analysis unscientific and meritless.
The other flaw in their “backwards event studies” is that they failed to perform any
statistical tests to determine if the results are statistically significant. There is no “null hypothesis”
being tested. Instead, they merely offer conclusions based on anecdotal evidence that the share
prices did or did not move on a specific day. This methodology is unscientific and their
observations cannot be extrapolated into meaningful implications for the companies at issue.
It is within the context of the problematic event study methodologies used by
Defendants’ expert that I performed my Binomial Test, which is a widely-used and generally
accepted statistical tool.35 As I explained in my Rebuttal Report, my binomial test was an attempt
to rectify Mr. Fischel and Dr. Grenadier’s methodological errors by providing an actual testable
hypothesis. Because their studies have already been performed, attempting to re-select individual
35
Introduction to Mathematical Statistics, by Robert V. Hogg, Joseph W. McKean, and Allen T. Craig, 6th Edition,
Pearson Prentice Hall, 2005.
22
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 26 of
28
events to examine would subject the analysis to the same unscientific and biased backwards event
study approach their studies suffer from. To execute a more scientifically sound event study, I
begin by treating all days on which either Mr. Fischel or Dr. Grenadier identified as days on which
a news article was published as a “news day”. As long as either Mr. Fischel or Dr. Grenadier
identified a news day during the overlapping period of their study, I considered the day of that
article’s release as a “news day” for purposes of my study.
I identified news days considered by Mr. Fischel for each of the Seven Affected
Companies by including those news days that Mr. Fischel identified in Appendix D of the Fischel
Report, titled “Summary of Company Specific News”.36 For Dr. Grenadier, I identified news days
for the Seven Affected Companies by including those news days that Dr. Grenadier has identified
in Appendix E of the Grenadier Report, titled “Potentially Value-Relevant Information During
the Run-Up Period 1/21/21 – 1/27/21.”37
Through this process, I am able to separate the trading days for each of the Seven
Affected Companies during the overlapping period of January 21, 2021 through January 27, 2021
into two distinct categories of “news” and “no-news” events, thereby generating a selection
criteria that isolates a test sample of “news” days.
After the “news” days were identified, I utilized a statistical testing tool called a
binomial test to assess whether the security prices of the Seven Affected Company reacted more
often to new days than one would expect from a random sample of days. As I showed in my
Rebuttal Report, for all Seven Affected Companies except TRVG, the Binomial Test finds that
the frequency of statistically significant price movements on days with Company-specific news
36
Fischel Report dated February 16, 2023, Appendix D.
37
Grenadier Report dated February 16, 2023, Appendix E.
23
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 27 of
28
is too unlikely to be considered random. I concluded that for AMC, BB, BBBY, GME, EXPR,
and NOK, the prices of these companies’ securities did react to company-specific information,
demonstrating market efficiency.
The binomial test is particularly useful in this case as it is a non-parametric test,
meaning there are no underlying assumptions made about the distribution of news days’ returns.
This element of the binomial test is important due to the extraordinarily short period of time being
examined. Other statistical testing techniques that rely on assumed distributions can yield
unreliable results when the number of testing observations is so low, as is the case here.38
As I showed in my Rebuttal Report, for all Seven Affected Companies except
TRVG, the Binomial Test finds that the frequency of statistically significant price movements on
days with Company-specific news is too unlikely to be considered random.
Defendants now claim that my Binomial Test did not go far enough, and that I
should have also conducted a Binomial Test on the frequency of statistically significant “non-
news” days.39 To this end, Defendants claim that a Binomial Test on statistically significant non-
news days would show that the stock price behavior of BB, BBBY, EXPR, and GME is
effectively no different on non-news days as compared to news days.40 However, Defendants’
definition of non-news days is biased by their requirement that only Dr. Grenadier’s definition of
“valuation relevant” should be considered and that news days identified by Mr. Fischel’s should
now be ignored. This discrepancy between Mr. Fischel and Dr. Grenadier demonstrates the
38
See, e.g., Nonparametric Statistical Methods, by Myles Hollander, Douglas A. Wolfe, and Eric Chicken, 3rd ed.
John Wiley & Sons, 2014, p. 505.
39
Defendants’ Daubert Motion, p. 18.
40
Dep. Ex. 288, Analysis in Response to Werner Table 7.
24
Case 1:21-md-02989-CMA Document 582-4 Entered on FLSD Docket 06/21/2023 Page 28 of
28
inherent subjectivity of identifying “valuation-relevant” days. Defendants’ binomial test on “non-
news” days is undermined by this inherent subjectivity.
Dated: June 21, 2023
25
File and source
- File
- gov.uscourts.flsd.590042.582.4.pdf
- Size
- 1,231,862 bytes
- SHA-256
- 4ada1715d61e7320dafe137591aadbd431422fe0e12785c015b5100c8a0c3987
- Original
- archive.org