Elastic Net Min-Max Normalization

[j.galimberti]

I am trying to understand EViews 12 ENet Min-Max regularization.

When I estimate using the built-in Min-Max option I get the following results:

Code: Select all

Method: Elastic Net Regularization Sample: 1998Q2 2022Q4 Included observations: 96 Penalty type: Elastic Net (alpha = 0.5) Regressor transformation: Min-Max --------------------------------------------------- Lambda 0.1 --------------------------------------------------- Variable Coefficient --------------------------------------------------- C -1.739382 SG101 0.149029 SG102 1.108223 --------------------------------------------------- d.f. 2 L1 Norm 2.996634 R-squared 0.287458 ---------------------------------------------------

When I try normalizing the regressors manually and use None transformation I get a very different result:

Code: Select all

Method: Elastic Net Regularization Sample: 1998Q2 2022Q4 Included observations: 96 Penalty type: Elastic Net (alpha = 0.5) Regressor transformation: None --------------------------------------------------- Lambda 0.1 --------------------------------------------------- Variable Coefficient --------------------------------------------------- C 1.654839 (SG101-@MIN(SG101,"1998q2 2022q4"))/(@MAX(SG101,"1998q2 2022q4")-@MIN(SG101,"1998q2 2022q4")) 1.060768 (SG102-@MIN(SG102,"1998q2 2022q4"))/(@MAX(SG102,"1998q2 2022q4")-@MIN(SG102,"1998q2 2022q4")) 1.820301 --------------------------------------------------- d.f. 2 L1 Norm 4.535908 R-squared 0.177011 ---------------------------------------------------

Note I kept lambda constant in order to avoid any potential difference from model selection, though it shouldn't cause any difference either.
I also tried pre-normalizing the dependent variable, and the discrepancies remain. Again, pre-normalizing the dependent variable shouldn't have any impact anyway.

So, how exactly is EViews 12 applying the Min-Max transformation?

[EViews Glenn]

Can you post your workfile?

[j.galimberti]

I can't as these data are not public. But I can replicate the issue with public data. See attachment. This file contains US CPI inflation, unemployment rate, and non-farm payrolls (an % chg).

Running the Elastic Net using the built-in Min-Max regularization (equation eq_auto_transform in the datafile):

Code: Select all

Dependent Variable: CPIAUCSL_PCA Method: Elastic Net Regularization Date: 03/07/25 Time: 10:37 Sample: 1970Q1 2024Q4 Included observations: 220 Penalty type: Elastic Net (alpha = 0.5) Regressor transformation: Min-Max Lambda 0.01 Variable Coefficient UNRATE -0.515604 PAYEMS_PCA 0.289640 C 6.694158 d.f. 2 L1 Norm 7.499401 R-squared -0.072951

Running with the manual regularization (equation eq_manual_transform in the datafile):

Code: Select all

Dependent Variable: CPIAUCSL_PCA Method: Elastic Net Regularization Date: 03/07/25 Time: 10:36 Sample: 1970Q1 2024Q4 Included observations: 220 Penalty type: Elastic Net (alpha = 0.5) Regressor transformation: None Lambda 0.01 Variable Coefficient (UNRATE-@MIN(UNRATE,"1970q1 2024q4"))/(@MAX(UNRATE,"1970q1 2024q4")-@MIN(UNRATE,"1970q1 2024q4")) 0.000000 (PAYEMS_PCA-@MIN(PAYEMS_PCA,"1970q1 2024q4"))/(@MAX(PAYEMS_PCA,"1970q1 2024q4")-@MIN(PAYEMS_PCA,"1970q1 2024q4")) 4.502401 C 1.010338 d.f. 1 L1 Norm 5.512739 R-squared 0.029680

As you can see, the estimates are very different even though they are supposedly using the same transformation.

[EViews Glenn]

Thanks for putting that together. It helps for us have a common set of data with which to work.

It is unsuprising that you were unable to replicate the results from the in-built EViews 12 scaled elastic net as the documentation is frustratingly silent about the approach. I apologize for that. As you know, elastic net is unlike simple OLS is that scaling and recentering of regressors has a substantive impact on estimates since the penalty function depends on levels and squares of the coefficient values. Matching results without knowing what is being done is quite difficult.

That said, EViews 12 computes the min-max normalization by first demeaning all variables, then estimating the model on the demeaned data using the transformation as you specified. The following programming code will estimate the model using original data with automatic scaling and using externally demeaned data:

Code: Select all

!alpha = .5 !lambda = 0.01 %xtrans = "minmax" series dcpiauc = cpiaucsl_pca - @mean(cpiaucsl_pca) series dunrate = unrate - @mean(unrate) series dpayem = payems_pca - @mean(payems_pca) !scale_1_13 = @max(dunrate) - @min(dunrate) !scale_2_13 = @max(dpayem) - @min(dpayem) ' elastic net ' (a) built-in equation eq1_a.enet(lambda=!lambda, xtrans={%xtrans}, alpha=!alpha) cpiaucsl_pca unrate payems_pca c ' (b) external transformation EViews 12/13 equation eq1_b.enet(lambda=!lambda, xtrans=none, alpha=!alpha) dcpiauc (dunrate - @min(dunrate))/!scale_1_13 (dpayem - @min(dpayem))/!scale_2_13 scalar eq1_b_c = @mean(cpiaucsl_pca) - eq1_b.c(1)*@mean(unrate) - eq1_b.c(2)*@mean(payems_pca)

Note that the reported coefficient results are for the demeaned and scaled data.

I hope that this answers your question. Please let me know if you have additional questions.

----

A last note...

While I try not to use forum responses to plug EViews updates, at this point I would like to point out that EViews 14 elastic net offers substantive improvements of the earlier routines. In addition to new, efficient estimation algorithms, there are a number of new specification and estimation options, and improved tools for analyzing and working with results. Note also that the approach to automatically scaling data has changed somewhat in EViews 14, as scales are computed using original data, and coefficient estimates are always presented in the original scale, simplifying interpretation of results.

[j.galimberti]

Yes, that worked, thanks!