ASREG: A powerful package for regressions in Stata

Introduction

asreg is a Stata program, written by Dr. Attaullah Shah. The program is available for free and can be downloaded from SSC by typing the following on the Stata command window:

ssc install asreg

asreg was primarily written for rolling/moving / sliding window regressions. However, with the passage of time, several useful ideas were conceived by its creator and users. Therefore, more features were added to this program. The primary uses of asreg can be summarized under the following three headings:

Speed Efficiency

asreg is an order of magnitude faster than estimating rolling window regressions through conventional methods such as Stata loops or using the Stata’s official rolling command. asreg has the same speed efficiency as asrol. All the rolling window calculations, estimation of regression parameters, and writing of results to Stata variables are done in the Mata language. Similarly, estimating Fama and MacBeth(1973) regression through asreg is several times faster than other available options.

Uses

asreg can be used for the following purposes. Click the following links for further details.

1.Rolling window regression

1.1 Rolling window betas

2. Recursive window regressions

3. by-group regressions

3.1 by-groups regression

3.2 by-groups regressions with residuals and fitted values

4. Fama-MacBeth (1973) regressions

4.1 FMB Regressions

4.2 FMB regression – what, how and where

4.3 FMB regressions with 25-portfolios – An example

5. Rolling window regression with Newey-West standard errors

6. Rolling window fitted values and residuals

7. Getting rolling window t-statistics

Questions / Answers

Question: How can I get similar results as produced by the Stata’s rolling command
Answer: See a detailed answer here in this post

Attaullah Shah2020-09-26T22:30:10+05:00December 8th, 2017|Uncategorized|41 Comments

41 Comments

Michel Steenhoven October 2, 2018 at 10:16 pm - Reply

I am currently trying to use the command asreg to estimate rolling beta’s for a large panel data set. I believe everything is working fine but as the dataset is quite large (260000 monthly observations) the computation takes a long time. I was wondering if there is a possibility to track the computation process, to be sure the command is actually processing.

Thank you in advance for your time and effort, kind regards,
Attaullah Shah October 12, 2018 at 4:10 pm - Reply

Dear Michel
I think 260000 observations should not take that much time unless your system is slow. As a suggestion, I have noted your point and shall consider this option in the next update.
Yang Wenhao November 1, 2018 at 1:18 am - Reply
First thank you very much for developing this wonderful package “asreg”, which is really helpful for many empirical researchers. Recently I used this package for running “Fama_Macbeth” Regressions, and I discovered some issues. When I run a regression using the command
```
asreg y x1 x2 x3 x4, fmb save(firststage)
```
it stores the first stage coefficients into “firststage”. But when I look into the “firststage”, I can see all the numbers are “correct”, but the variable names are somehow mismatched. For example, If I simply compute the average of all the coefficients, it should be identical to the reported 2nd stage results. But instead, I get the average coefficient of “x4” is actually recorded in “_b_x1”. And this problem does not always happen. In my case, the right-hand side variables are “aret” and “turn_ratio” . Not sure if it’s due to the specific naming of these variables.
- Attaullah Shah November 6, 2018 at 2:11 am - Reply
  
  Dear Wenhao
  Thanks for your feedback. I tested asreg on several datasets but did encounter the problem you are referring to. I would appreciate if you share a sample data that produces the said error.
Mohammad Alhasham November 6, 2018 at 2:08 am - Reply

hope this email finds you well. I have question regarding your asreg stata command.

I would like to run FM regression with firm and time fixed effects. I was able to do so by adding xi: before asreg. However, the option absorb is not allowed.

My question is there a way to run FM regression with asreg and also not display or report the dummy variables for firm and time fixed effect.
- Attaullah Shah November 9, 2018 at 12:30 pm - Reply
  
  Alhasham: Option absorb is not currently supported by asreg.
Austin Moss November 9, 2018 at 12:32 pm - Reply
I am attempting to use your asreg command in STATA to estimate firm and industry expected return on equity. I have used the command successfully to calculate it by industry. However, when I use the same code, but switch firmid for industryid the code will run for some of the data, but then I get an error:
```
bys firmfe: asreg retovrrf mktrf smb hml, wind(mnth 60) min(24) fit 
ASREG4slfl(): 3200 conformability error
ASREGW():    -   function returned error
             -   function returned error
```
I would greatly appreciate some guidance on what might be causing this error.
- Attaullah Shah November 9, 2018 at 12:36 pm - Reply
  Austin Moss:
  Thanks for your feedback. I would appreciate if you can tell which version are you using. You can find that by typing
  
  which asreg
  
  Second, if you can kindly share a sample of data that can generate the above error, I shall fix the error ASAP.
Michel Steenhoven November 9, 2018 at 12:40 pm - Reply

Thank you for your response, unfortunately I got quite sick a few days ago so I haven’t been able to respond. I believe there was something wrong with the system indeed. I ran the regression again at a later stage and it was done quite quickly.
Austin Moss November 9, 2018 at 12:45 pm - Reply

I have attached my code and the first 150,000 observations of my sample. When I run the code as-is with irrelevant stuff highlighted out the code will calculate the ‘fit’ variables until observation #125,247. After that everything is blank/missing. This is what is giving me the biggest problem–why will it work for some of my data, but not the rest?

The data set attached is the result of running my code. To re-run the asreg code, just run the drop line under the unhighlighted out asreg code and then just run the asreg line.

The two ‘asreg’ code blocks above the currently non-highlighted one work for industryid (industryfe) but not firmid (gvkey or firmfe). However, the error usually occurs later in the data set and would be hard to transfer the whole data set via email. I am hoping that the solution for the above will also be the solution for the other blocks of code.
- Attaullah Shah December 10, 2018 at 10:27 pm - Reply
  Austin Moss: asreg version 3.2 is now available for download from SSC (thanks to Kit Baum). This version fixes a minor bug in one of the Mata functions. Mata function ASREG4s0f1() would throw an error if the option fit was used on a dataset that did not have enough observations for estimating a regression. To update,
  
  ssc install asreg, replace
Vince December 4, 2018 at 11:07 am - Reply
Dear Dr. Shah:
I am trying to apply your asreg command using Fama-Macbeth.
```
asreg indep   dep1 dep2  dep3, fmb newey(3)
```
Is it possible in your regression to test the equality of the coefficients for dep1=dep2.
I know that if you use the regressioStatamand in stata you can test whether dep1=dep2.
How would you advise to do the dep1=dep2 in a Fama-Macbeth setting?
Thanks much,
- Attaullah Shah December 10, 2018 at 10:25 pm - Reply
  
  Vince: I have not tested equality of coefficients before and hence cannot comment on this.
Vince December 10, 2018 at 10:28 pm - Reply

Thank you sir. How about reporting the standard errors and the degrees of freedom from the first stage. Is it possible to extract them?
Chris H Zhang March 27, 2019 at 10:48 am - Reply
Hi professor,

I’m from STATALIST and would like to ask you a question regarding “asreg”.

I used an old version of asreg function (used it roughly a year ago), to calculate by each stock(index) the residual value of regression from a panel data.
```
bysort index: asreg log_QSpread t, fit

rename _residuals log_QSpread_notrend

label variable log_QSpread_notrend "log_QSpread_notrend"
```
Above us basically what I did.

I recently realized that the function has been updated. I don’t know what version I used back then to execute the above codes. just wondering whether the following updates would affect the results I got? Do I have to redo everything I did?

Thanks
- Attaullah Shah March 27, 2019 at 10:49 am - Reply
  
  Chris H Zhang
  The updates to asreg have not changed anything fundamental. The calculations of the regression coefficients remain unchanged, therefore, the calculations of the older and newer versions of asreg are still the same.
Dimitri April 15, 2019 at 10:07 pm - Reply

Dear Sir,

Thank you for contributing asreg to the community. Is there any immediate plans to implement weighted FM regressions using aweight? Xtfmb supports weighted FM regressions, but asreg does not.

Given that asreg is much faster than xtfmb, it would be wonderful if you could add the functionality to your next release. Any chance of seeing this in the very near future?

Thank you.
Weighted FM June 3, 2019 at 12:40 am - Reply

Do you have any plans to implement weighted Fama-MacBeth into the package?
- Attaullah Shah June 9, 2019 at 5:57 pm - Reply
  
  Thanks for asking. However, there is no immediate plan for adding weights to FMB regression.
Adj. R2 June 8, 2019 at 1:59 am - Reply

Is the R2 given for FMB adjusted or unadjusted? If unadjusted, do you have plans to include an option for the adjusted R2?
- Attaullah Shah June 9, 2019 at 5:38 pm - Reply
  The r-squared is the average r-squared from the first-stage cross-sectional regressions. Upon your request, I have also added average adjusted r-squared, which is again the average from the first stage cross-sectional regressions. You can download the beta version by typing the following line in Stata. The new version will be available on SSC after a week or two.
  
  net install asreg, from(http://fintechprofessor.com) replace
  
  Please note that you have to copy and past the full line in Stata.
Anonymous June 13, 2019 at 9:02 am - Reply

Hi,

I hope this message finds you well.

When trying to get the adj-R2 using the beta version, I ran the same command “asreg varlist, fmb newey(#)” as before, but stata returned with an error (please see below). Should the command be different in the beta version?

stata(): 3598 Stata returned error
FMB3(): – function returned error
: – function returned error

Thank you so much
- Attaullah Shah June 13, 2019 at 12:06 pm - Reply
  
  To debug the issue, can you please send me a sample of your dataset that generates the said error. You can email me at attaullah.shah@imsciences.edu.pk
Adj. R2 June 14, 2019 at 11:56 am - Reply

Thanks for the inclusion of the FM Adj. R2. One issue I noticed is that when outputting regressions via eststo and esttab, the ar2 is blank in the output table. The Adj. R2 is displayed perfectly in the within Stata output.

Also, is implementing weighted regressions in Mata computationally difficult? Currently, the best substitute for weighted FM regressions, xtfmb, takes minutes to run for monthly observations, while equal weighted FM in asreg runs lightning fast in seconds for the same dataset.

Thanks
- Attaullah Shah June 14, 2019 at 12:04 pm - Reply
  I have not used estout package, I use asdoc instead for outputting my tables
  
  ssc install asdoc
  
  asreg returns the r2 and adj. R2 in e() macro and can be accessed by typing ereturn list. See
  
  eret list scalars: e(N) = 200 e(N_g) = 20 e(df_m) = 2 e(df_r) = 19 e(F) = 195.0437518440737 e(r2) = .8369070768356324 e(adjr2) = .8165204703807831
  
  Currently, I lack the motivation to write the weighted FMB.
shazil September 2, 2019 at 10:27 pm - Reply
hi, I am working on panel data with 17 industries and year 2005 to 2017. I used asreg using the following command:
```
bys year inds_code: asreg TACC2_w term1_w term2j_w term3_w, noconstant min(10) fit
```
I compared the residuals I got from asreg with the residuals I got from running the following regression. I see there a little difference between the residuals from both sources for the year 2006 and industry(1100)
```
reg TACC2_w term1_w term2j_w term3_w, vce(cluster ISIN), if inds_code== 1100 & year==2006
predict r if inds_code== 1100 & year==2006, resid
```
could you please explain to me why is this so?
- Attaullah Shah September 2, 2019 at 11:59 pm - Reply
  
  Shazil
  In the second command where you are using reg, you have not used the noconstant option. Try that and let me know.
shazil September 3, 2019 at 3:23 pm - Reply

my apologies and thank you so much, it worked
Shaazil October 14, 2019 at 9:13 pm - Reply
Hi professor,
I am working on panel data, and I am running asreg by Industry and year, I have a few factor variables, how can I use them in asreg. I am not sure if it will work if use i.var in asreg the way we use in panel data regression?
```
bys Industry: asreg dep-var var1 var2 i.var3
```
My var3 is a factor or categorical variable for example.
- Attaullah Shah October 16, 2019 at 2:01 am - Reply
  Shazil
  You can first convert factor variable to dummy variables with tab command and then use all those created variables with asreg, see this example
  
  sysuse auto tab rep78, gen(rep) reg price rep2 rep3 rep4 rep5 headroom trunk weight length foreign reg price i.rep78 headroom trunk weight length foreign
  
  The above two regression yield the same results. So the asreg example would be:
  
  asreg price rep2 rep3 rep4 rep5 headroom trunk weight length foreign
Anonymous October 16, 2019 at 7:35 pm - Reply

thanks alot, i will try
Regards
Shazia QADAM
Andrea November 30, 2019 at 3:01 pm - Reply

Hi
is there the possibility to run asreg with lead and lag operators?

Thanks in advance
- Attaullah Shah November 30, 2019 at 3:15 pm - Reply
  Andrea
  Yes, this is possible. However, we need to create the variables first before using them in asreg. So I am creating two lead and two lagged variables in the following example:
  
  webuse grunfeld, clear **First create the lead variables gen Finvest = F.invest gen F2invest = F2.invest **Now two lagged variables gen Lkstock = L.kstock gen L2kstock = L2.kstock **use them in asreg bys company: asreg mvalue Finvest F2invest Lkstock L2kstock, window(year 15)
Christian Weber December 13, 2019 at 5:15 pm - Reply

Dear Attaullah,

thank you for providing this program. Somehow, I get weird results though. Because the number of observations used does not correspond to what I specified in wind(). For example when I run the following

foreach var of varlist var1-var2{
bysort firm: asreg ret`var’, wind(date 400)
rename _Nobs _Nobs_`var’
drop _R2 _adjR2 _b_cons
}

I get at most 274 observations, when I used (wind(252), I got 174 observations.

Can you help me out? Thank you so much!

Best,

Christian
- Attaullah Shah December 13, 2019 at 5:26 pm - Reply
  Christian Weber
  The reason you are getting fewer observations than what you specify in the option window() is that asreg uses actual available observations that are available within the given range, not the one that you specify in the window. Consider the following example where we have data for 10 years, but some of them are missing
  
  Year ---- 2001 2002 2004 2006 2007 2008 2010
  
  For this dataset, if you use option window as window(year 10), asreg will find only 7 observations when it estimates the regression for the last observation. In other words, the above option is interpreted as all data points that are within the 10-years range. It does not mean include 10 observation, rather it means that data pertaining to the years 2010, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, and 2001. If some of these years are missing, asreg will show fewer than 10 observations.
George Ford January 22, 2020 at 9:17 am - Reply

This program is a life saver.

Just to get an idea of the speed difference. I am estimating 1,040 two-year Betas for 1,492 stocks. Stata’s -rolling- command took 373 seconds (6.2 minutes) to estimate the Betas for a SINGLE stock (15MP4; Ryzen 2700x; 32GB Ram; M.2 drives). It would have taken 6.4 days to estimate all of it. In contrast, -asreg- estimated it for all the stocks in 189 seconds (3 minutes).

You are my hero.

Note that -asreg- would not run in Version 16, but I did get it run without difficulty in Version 15.
Attaullah Shah January 22, 2020 at 12:14 pm - Reply

George Ford
Thanks for the kind words and your feedback. Can you please specify what error do you get on Stata 16.0
George Ford January 22, 2020 at 9:54 pm - Reply

: 3499 ASREGW() not found.
YA January 27, 2020 at 9:29 pm - Reply

Hi,

Many thanks for your awesome command.

1. How I can create a table that presents a time series means of the coefficient estimates from the yearly regressions using asreg.

2. The table uses Newey-West adjustment with three lags to calculate p-values and reports them in parentheses

3. The significance levels are for a test of the hypothesis that the time series mean is equal to zero, using the time series standard error of the mean estimate for each coefficient.

George Ford January 31, 2020 at 6:40 pm - Reply

I found this online.

sysuse auto, clear
mata:
void i_crit(todo,b,crit,g,H)
{
	external y,X,W
	m=X'(y-X*b')
	crit=(m'*W*m)
}
y = st_data(., "price")
cons=J(rows(y),1,1)
X = st_data(., "weight length"), cons
W=cholinv(X'X)
init=J(1,cols(X),0)
S=optimize_init()
// build the constraint matrix
C = (1, 0, 0)
c = (0)
Cc = (C, c)
optimize_init_constraints(S,Cc)
optimize_init_evaluator(S, &i_crit())
optimize_init_which(S,"min")
optimize_init_evaluatortype(S,"d0")
optimize_init_params(S,init)
p=optimize(S)
p
end
constraint 1 weight = 0
cnsreg price weight length, constraints(1)

Would it be difficult to add contrained regression -asreg- ?

Attaullah Shah February 2, 2020 at 8:49 am - Reply

George Ford
Yes, it would be difficult to add this routine to asreg.

ASREG: A powerful package for regressions in Stata