Mixed logit models in practice

8 posts / 0 new
Last post
Peter Davidson
Mixed logit models in practice

There's been lots of academic discussion over the last 20 years on the benefits of mixed logit models, and how they overcome the problems with the widely used hierarchical logit models. But I've been looking for good examples of the use of mixed logit models in practice. Does anyone know of any practical (city-scale or larger) transport models that make use of mixed logit? Any published information? Also would be useful to know if anyone has considered their use but decided against it, and what their reasons

were.

winufuwub

This question is interesting. There is no doubt that for short-term demand
model predictions, mixed logit models are the standard as they are able to
tackle all the potential deficiencies of simpler model structures such as
nested logit (i.e., cross-correlation, heteroscedasticity, panel data
considerations). There are plenty of reports detailing this.
Notwithstanding, for practical long-term forecasting, where there is a need
for supply-demand equilibration, it is obviously more difficult to think
about using a complex model structure such as mixed logit, unless you are
applying a fully disaggregate approach; however, the examples I know that
have done this, suffer precisely from the fact that it is not clear how you
achieve/guarantee supply-demand equilibration in that case.
Thus, I am not aware of a "practical" study such as Peter is asking for;
the reasons for this are not clear, but it could partly be due to the fact
that, sadly, the average consultant dealing with "practical applications"
is typically years behind the state-of-the art.

Juan de Dios

J. de D. Ortúzar
Emeritus Professor
Department of Transport Engineering and Logistics
Pontificia Universidad Catolica de Chile
www.ing.puc.cl/jos

El mar, 31 may 2022 a las 7:36, Peter Davidson ()
escribió:

> There's been lots of academic discussion over the last 20 years on the
> benefits of mixed logit models, and how they overcome the problems with the
> widely used hierarchical logit models. But I've been looking for good
> examples of the use of mixed logit models in practice. Does anyone know of
> any practical (city-scale or larger) transport models that make use of
> mixed logit? Any published information? Also would be useful to know if
> anyone has considered their use but decided against it, and what their
> reasons
>
> were.
> --
> Full post: https://tmip.org/content/mixed-logit-models-practice
> Manage my subscriptions: https://tmip.org/mailinglist
> Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800
>

kywyjusin

I think mixed logit would make an excellent case study on technical
transfer in the travel modeling realm, perhaps aside nested logit as a
successful counter factual.

As an "average consultant" myself, my reflex to Prof Ortúzar's hypothesis
that the lack of use of mixed logit is "... due to the fact that, sadly,
the average consultant dealing with 'practical applications' is typically
years behind the state-of-the-art" is to suggest a parallel hypothesis
operating in the reverse direction: it may be due to the fact that, sadly,
the average academic dealing with 'theoretical research' typically has a
poor understanding of the state-of-the-practice. There's probably evidence
to support both hypotheses, but leaving them unexplored does not move us
forward. It takes both academics and practitioners for successful tech
transfer to occur. Why has it not happened for mixed logit? Perhaps we
could start this work via a discussion on TMIP.

As a practitioner, I do not have a firm understanding of where the
literature stands on this matter. My questions are:

1. Has the research community demonstrated that mixed logit makes more
accurate predictions across time and/or space than multinomial logit? Or
nested logit? Can academics on this listserv share the two or three key
references on this front?

2. Have the key experiments demonstrating improved prediction (as
identified in 1) been successfully replicated by independent research
teams? Can academics on this listserv share references to this evidence?

3. Has the research community demonstrated these methods can be efficiently
implemented in the open source software, such as ActivitySim, that the
practitioner community has developed, in part, for the purpose of tech
transfer? Where can I find these references?

4. As a practitioner, I see (1), (2), and (3) as table stakes for
successful tech transfer, i.e., demonstrated success, confirmation of
success, and computationally viable implementation. Are these standards
reasonable? Too high? Too low?

thanks,
David Ory

On Wed, Jun 1, 2022 at 11:01 AM winufuwub wrote:

> This question is interesting. There is no doubt that for short-term demand
> model predictions, mixed logit models are the standard as they are able to
> tackle all the potential deficiencies of simpler model structures such as
> nested logit (i.e., cross-correlation, heteroscedasticity, panel data
> considerations). There are plenty of reports detailing this.
> Notwithstanding, for practical long-term forecasting, where there is a need
> for supply-demand equilibration, it is obviously more difficult to think
> about using a complex model structure such as mixed logit, unless you are
> applying a fully disaggregate approach; however, the examples I know that
> have done this, suffer precisely from the fact that it is not clear how you
> achieve/guarantee supply-demand equilibration in that case.
> Thus, I am not aware of a "practical" study such as Peter is asking for;
> the reasons for this are not clear, but it could partly be due to the fact
> that, sadly, the average consultant dealing with "practical applications"
> is typically years behind the state-of-the art.
>
> Juan de Dios
>
> J. de D. Ortúzar
> Emeritus Professor
> Department of Transport Engineering and Logistics
> Pontificia Universidad Catolica de Chile
> www.ing.puc.cl/jos
>
> El mar, 31 may 2022 a las 7:36, Peter Davidson ()
> escribió:
>
> > There's been lots of academic discussion over the last 20 years on the
> > benefits of mixed logit models, and how they overcome the problems with
> the
> > widely used hierarchical logit models. But I've been looking for good
> > examples of the use of mixed logit models in practice. Does anyone know
> of
> > any practical (city-scale or larger) transport models that make use of
> > mixed logit? Any published information? Also would be useful to know if
> > anyone has considered their use but decided against it, and what their
> > reasons
> >
> > were.
> > --
> > Full post: https://tmip.org/content/mixed-logit-models-practice
> > Manage my subscriptions: https://tmip.org/mailinglist
> > Stop emails for this post:
> https://tmip.org/mailinglist/unsubscribe/13800
> >
> --
> Full post: https://tmip.org/content/mixed-logit-models-practice
> Manage my subscriptions: https://tmip.org/mailinglist
> Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800
>

winufuwub

1. My English betrayed me a bit here David, my intention was not to claim
that the *average consultant* was - perhaps - behind the state of the art
because I have no idea if that is the case at all ... I was thinking of the
average *Large Consulting Firm* - who always amazed me for their lack of
interest on improving on "usual practice" (when I was involved in those
matters). By the way, the case was even worse some years ago with the
average city's or state planning office, who were very reluctant to invest
in new and more contemporary models and methodologies.
2. Re your questions: (1) Has the research community demonstrated that
mixed logit makes more accurate predictions across time and/or space than
multinomial logit? Or nested logit? Can academics on this listserv share
the two or three key references on this front?
This is a difficult question to answer as, in my opinion, it is not
correctly formulated. Multinomial logit (MNL) and nested logit (NL) make
very strong assumptions about behaviour that mixed logit (ML) does not
require; therefore, in cases where these two models' assumptions do not
hold, it can be easily shown that their predictions can be very biased
indeed; a well-known paper by Huw Williams and myself shows this very
nicely (Williams, H.C.W.L and Ortúzar, J . de D. (1982) behavioural
theories of dispersion and the mis-specification of travel demand models.*
Transportation Research Part B: Methodological** 16*, 167-219).
(2) Have the key experiments demonstrating improved prediction (as
identified in 1) been successfully replicated by independent research
teams? Can academics on this listserv share references to this evidence?
Again, who would be interested in making a comparison between the
prediction of a wrongly specified model and an appropriate one? It is so
obvious that no journal would publish it.
(3) Has the research community demonstrated these methods can be
efficiently implemented
in open source software, such as ActivitySim, that the practitioner
community has developed, in part, for the purpose of tech transfer? Where
can I find these references?
The answer to this question is "not that I know".
(4) As a practitioner, I see (1), (2), and (3) as table stakes for
successful tech transfer, i.e., demonstrated success, confirmation of
success, and computationally viable implementation. Are these
standards reasonable?
Too high? Too low?
I do not agree with (1) and (2), as already stated, but (3) is a fair
challenge, and - in my opinion - of adequate standard. I am not familiar
with ActivitySim and I have not been involved in large-scale planning
applications for a long while, so I do not think I am qualified to even
guess an answer. But if the problem you are trying to tackle involves
correlated and cross-correlated alternatives and utility functions that are
highly heteroscedastic, your results - if your model is of MNL or NL form -
should be highly suspect.

Juan de Dios

El jue, 2 jun 2022 a las 13:12, kywyjusin () escribió:

> I think mixed logit would make an excellent case study on technical
> transfer in the travel modeling realm, perhaps aside nested logit as a
> successful counter factual.
>
> As an "average consultant" myself, my reflex to Prof Ortúzar's hypothesis that
> the lack of use of mixed logit is "... due to the fact that, sadly, the
> average consultant dealing with 'practical applications' is typically years
> behind the state-of-the-art" is to suggest a parallel hypothesis operating
> in the reverse direction: it may be due to the fact that, sadly, the
> average academic dealing with 'theoretical research' typically has a poor
> understanding of the state-of-the-practice. There's probably evidence to
> support both hypotheses, but leaving them unexplored does not move us forward.
> It takes both academics and practitioners for successful tech transfer to
> occur. Why has it not happened for mixed logit? Perhaps we could start
> this work via a discussion on TMIP.
>
> As a practitioner, I do not have a firm understanding of where the
> literature stands on this matter. My questions are:
>
> 1. Has the research community demonstrated that mixed logit makes more accurate
> predictions across time and/or space than multinomial logit? Or nested
> logit? Can academics on this listserv share the two or three key references
> on this front?
>
> 2. Have the key experiments demonstrating improved prediction (as
> identified in 1) been successfully replicated by independent research
> teams? Can academics on this listserv share references to this evidence?
>
> 3. Has the research community demonstrated these methods can be efficiently implemented
> in the open source software, such as ActivitySim, that the practitioner
> community has developed, in part, for the purpose of tech transfer? Where
> can I find these references?
>
> 4. As a practitioner, I see (1), (2), and (3) as table stakes for
> successful tech transfer, i.e., demonstrated success, confirmation of
> success, and computationally viable implementation. Are these standards reasonable?
> Too high? Too low?
>
> thanks,
> David Ory
>
> On Wed, Jun 1, 2022 at 11:01 AM winufuwub wrote:
>
> > This question is interesting. There is no doubt that for short-term
> demand model predictions, mixed logit models are the standard as they are
> able to tackle all the potential deficiencies of simpler model structures
> such as nested logit (i.e., cross-correlation, heteroscedasticity, panel
> data considerations). There are plenty of reports detailing this.
> Notwithstanding, for practical long-term forecasting, where there is a need for
> supply-demand equilibration, it is obviously more difficult to think about
> using a complex model structure such as mixed logit, unless you are applying
> a fully disaggregate approach; however, the examples I know that have
> done this, suffer precisely from the fact that it is not clear how you achieve/guarantee
> supply-demand equilibration in that case.
> Thus, I am not aware of a "practical" study such as Peter is asking for;
> the reasons for this are not clear, but it could partly be due to the fact
> that, sadly, the average consultant dealing with "practical applications"
> is typically years behind the state-of-the art.
>
> Juan de Dios
> >
> > J. de D. Ortúzar
> > Emeritus Professor
> > Department of Transport Engineering and Logistics
> > Pontificia Universidad Catolica de Chile
> > www.ing.puc.cl/jos
> >
> > El mar, 31 may 2022 a las 7:36, Peter Davidson ()
> > escribió:
> >
> > > There's been lots of academic discussion over the last 20 years on the
> > > benefits of mixed logit models, and how they overcome the problems with
> > the
> > > widely used hierarchical logit models. But I've been looking for good
> > > examples of the use of mixed logit models in practice. Does anyone know
> > of
> > > any practical (city-scale or larger) transport models that make use of
> > > mixed logit? Any published information? Also would be useful to know if
> > > anyone has considered their use but decided against it, and what their
> > > reasons
> > >
> > > were.
> > > --
> > > Full post: https://tmip.org/content/mixed-logit-models-practice
> > > Manage my subscriptions: https://tmip.org/mailinglist
> > > Stop emails for this post:
> > https://tmip.org/mailinglist/unsubscribe/13800
> > >
> > --
> > Full post: https://tmip.org/content/mixed-logit-models-practice
> > Manage my subscriptions: https://tmip.org/mailinglist
> > Stop emails for this post:
> https://tmip.org/mailinglist/unsubscribe/13800
> >
> --
> Full post: https://tmip.org/content/mixed-logit-models-practice
> Manage my subscriptions: https://tmip.org/mailinglist
> Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800
>

Greg Erhardt

Peter,

Mixed logit is actively used in practice in the United States and has been for over a decade. It is used specifically to estimate value of time distributions for use in agent-based models. To my knowledge, this was first done in San Francisco in about 2009 in the SF-CHAMP model. As reported in Sall et al (2010):

"The mixed-logit time-of-day model was estimated based on a stated preference survey administered to 663 travelers driving to downtown San Francisco where they were asked to trade off cost, time shifts, and mode shifts. It is applied disaggregately to all auto-person trips based on each individuals' trip purpose and value of time. Mixed logit is important in this case because it allows the user to estimate a distribution on a coefficient, rather than just the mean value. In this case, a distribution was estimated on the travel time variable, asserting a lognormal form. The cost coefficients are estimated as standard, non-distributed, coefficients segmented by income."

Standard estimation software (we used ALOGIT) can be used to estimate mixed logit models. After estimating the distribution, it is implemented using Monte Carlo simulation. Each agent draws a personal value of time from the distribution and carries that value of time with them through the simulation as one additional attribute on the agent (Erhardt et al 2008).

When applying a choice model, we simply scale the time/cost coefficients according to that agent's value of time and then apply it as a multinomial logit or nested logit model. I suspect that most users would refer to this as using a distributed value of time rather than as using mixed logit. There may be a reason not to consider this implementation strategy "true" mixed logit, and I'd be interested in understanding the implications of that if someone can shed light. Nonetheless, it is a practical way to achieve an important benefit of mixed logit-capturing user heterogeneity--for one of the most important variables in our models.

I believe this strategy is now standard practice in the activity-based models used in practice in the United States. I'm not sure that the distributions are always re-estimated, or if they're instead updated based on local wage rates. Either way, the implementation is the same. Collectively, these models have been successfully used for hundreds of planning studies. You could also do this in a trip-based model, as long as the model is agent based, such as MITO. You can demonstrate that the model will produce a different result than using an average value of time, or an average segmented by income level. This matters more when the price being tested is high because the value of time distribution tends to have a long tail-a relatively small number of people willing to pay a lot because they really don't want to be late. Maybe you're trying to catch a flight or pick up your kid before daycare closes.

References are below. If they're still available for download, I'm happy to send a copy.

Erhardt, GD, B Charlton, J Freedman, J Castiglione, and M Bradley. "Enhancement and Application of an Activity-Based Travel Model for Congestion Pricing." Presented at the Transportation Research Board Innovations in Travel Modeling Conference, Portland, Oregon, 2008.

Sall, Elizabeth, Elizabeth Bent, Jesse Koehler, Billy Charlton, and Gregory D Erhardt. "Evaluating Regional Pricing Strategies in San Francisco--Application of the SFCTA Activity-Based Regional Pricing Model." Presented at teh 89th Transportation Research Board Annual Meeting, Washington, D.C., 2010.

Kind Regards,
Greg

Greg Erhardt
Associate Professor of Civil Engineering
University of Kentucky
261 Oliver H. Raymond Bldg., Lexington, KY 40506
859-323-4856, greg.erhardt@uky.edu
transportlab.net

From: peter=transposition.com.au@mg.tmip.org On Behalf Of Peter Davidson
Sent: Tuesday, May 31, 2022 4:35 AM
To: TMIP
Subject: [TMIP] Mixed logit models in practice

CAUTION: External Sender

There's been lots of academic discussion over the last 20 years on the benefits of mixed logit models, and how they overcome the problems with the widely used hierarchical logit models. But I've been looking for good examples of the use of mixed logit models in practice. Does anyone know of any practical (city-scale or larger) transport models that make use of mixed logit? Any published information? Also would be useful to know if anyone has considered their use but decided against it, and what their reasons

were.
--
Full post: https://tmip.org/content/mixed-logit-models-practice
Manage my subscriptions: https://tmip.org/mailinglist
Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800

Mark Bradley

Thanks to Peter, Juan de Dios, Dave and Greg for an interesting discussion.

Several of the activity-based models used in the US use the travel time and cost coefficient formulations developed in the report: "Improving Our Understanding of How Highway Congestion and Pricing Affect Travel Demand", TRB Strategic Highway Research Program (SHRP 2) Report S2-C04-RW-1 Download available at https://www.trb.org/Main/Blurbs/168141.aspx Peter Vovsha, Hani Mahmassani and I were PI's, and Hani estimated mixed logit models to get lognormally distributed time coefficients, similar to what Greg Erhardt described. (A 1993 TRB paper with similar models is Ben-Akiva, Bolduc and Bradley, "Estimation of Travel Choice Models with Randomly Distributed Values of Time" at https://onlinepubs.trb.org/Onlinepubs/trr/1993/1413/1413-010.pdf)

For more complex mixed logit models with several distributed parameters and (possibly) estimated correlation structures between random parameters, I would have two concerns for application.... (1) What are the run time implications of drawing from complex distributions?, and (2) When simulating over synthetic populations with millions of persons and tours, does the law of large numbers ensure that drawing just one set of parameters from the distribution(s) per choice is sufficient, or would using multiple sets of random draws per choice give more reliable results?

RSG has done a couple of applied studies in recent years using "integrated choice/latent variable" (ICLV) models estimated using mixed logit and containing several coefficients with random components and applied to large numbers of cases. One, ACRP Research Report 204 "Air Demand in a Dynamic Competitive Context with the Automobile" applied the ICLV model to all the long-distance tours generated for the full (synthesized) US population across a number of different future scenarios. (Download available at https://onlinepubs.trb.org/onlinepubs/acrp/acrp_rpt_204.pdf )

Another, NCRRP Report 4, "Intercity Passenger Rail in the Context of Dynamic Travel Markets" also applied an ICLV model in scenario analysis, but was made to be more user-accessible, applying it in Excel on a smaller sample of travelers. (https://nap.nationalacademies.org/catalog/22072/intercity-passenger-rail... )

Both of those examples are on the edge of research and practical consulting, and haven been used in practical applications elsewhere that I know of. (There are endogeneity issues in using hybrid models with latent variables in forecasting, which Caspar Chorus has written about (https://www.sciencedirect.com/science/article/abs/pii/S0967070X14001905), but that is separate from the more general question of using mixed logit models in applied model systems.

Mark

..........................................
Mark Bradley
Principal
Pronouns: he/him/his

RSG
524 Arroyo Ave| Santa Barbara, CA 93109
415.328.4766
www.rsginc.com

From: greg.erhardt=uky.edu@mg.tmip.org On Behalf Of Greg Erhardt
Sent: Friday, June 3, 2022 7:23 AM
To: TMIP
Subject: Re: [TMIP] Mixed logit models in practice

CAUTION - EXTERNAL EMAIL

Peter,

Mixed logit is actively used in practice in the United States and has been for over a decade. It is used specifically to estimate value of time distributions for use in agent-based models. To my knowledge, this was first done in San Francisco in about 2009 in the SF-CHAMP model. As reported in Sall et al (2010):

"The mixed-logit time-of-day model was estimated based on a stated preference survey administered to 663 travelers driving to downtown San Francisco where they were asked to trade off cost, time shifts, and mode shifts. It is applied disaggregately to all auto-person trips based on each individuals' trip purpose and value of time. Mixed logit is important in this case because it allows the user to estimate a distribution on a coefficient, rather than just the mean value. In this case, a distribution was estimated on the travel time variable, asserting a lognormal form. The cost coefficients are estimated as standard, non-distributed, coefficients segmented by income."

Standard estimation software (we used ALOGIT) can be used to estimate mixed logit models. After estimating the distribution, it is implemented using Monte Carlo simulation. Each agent draws a personal value of time from the distribution and carries that value of time with them through the simulation as one additional attribute on the agent (Erhardt et al 2008).

When applying a choice model, we simply scale the time/cost coefficients according to that agent's value of time and then apply it as a multinomial logit or nested logit model. I suspect that most users would refer to this as using a distributed value of time rather than as using mixed logit. There may be a reason not to consider this implementation strategy "true" mixed logit, and I'd be interested in understanding the implications of that if someone can shed light. Nonetheless, it is a practical way to achieve an important benefit of mixed logit-capturing user heterogeneity--for one of the most important variables in our models.

I believe this strategy is now standard practice in the activity-based models used in practice in the United States. I'm not sure that the distributions are always re-estimated, or if they're instead updated based on local wage rates. Either way, the implementation is the same. Collectively, these models have been successfully used for hundreds of planning studies. You could also do this in a trip-based model, as long as the model is agent based, such as MITO. You can demonstrate that the model will produce a different result than using an average value of time, or an average segmented by income level. This matters more when the price being tested is high because the value of time distribution tends to have a long tail-a relatively small number of people willing to pay a lot because they really don't want to be late. Maybe you're trying to catch a flight or pick up your kid before daycare closes.

References are below. If they're still available for download, I'm happy to send a copy.

Erhardt, GD, B Charlton, J Freedman, J Castiglione, and M Bradley. "Enhancement and Application of an Activity-Based Travel Model for Congestion Pricing." Presented at the Transportation Research Board Innovations in Travel Modeling Conference, Portland, Oregon, 2008.

Sall, Elizabeth, Elizabeth Bent, Jesse Koehler, Billy Charlton, and Gregory D Erhardt. "Evaluating Regional Pricing Strategies in San Francisco--Application of the SFCTA Activity-Based Regional Pricing Model." Presented at teh 89th Transportation Research Board Annual Meeting, Washington, D.C., 2010.

Kind Regards,
Greg

Greg Erhardt
Associate Professor of Civil Engineering
University of Kentucky
261 Oliver H. Raymond Bldg., Lexington, KY 40506
859-323-4856, greg.erhardt@uky.edu
transportlab.net

From: peter=transposition.com.au@mg.tmip.org On Behalf Of Peter Davidson
Sent: Tuesday, May 31, 2022 4:35 AM
To: TMIP
Subject: [TMIP] Mixed logit models in practice

CAUTION: External Sender

There's been lots of academic discussion over the last 20 years on the benefits of mixed logit models, and how they overcome the problems with the widely used hierarchical logit models. But I've been looking for good examples of the use of mixed logit models in practice. Does anyone know of any practical (city-scale or larger) transport models that make use of mixed logit? Any published information? Also would be useful to know if anyone has considered their use but decided against it, and what their reasons

were.
--
Full post: https://tmip.org/content/mixed-logit-models-practice
Manage my subscriptions: https://tmip.org/mailinglist
Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800
--
Full post: https://tmip.org/content/mixed-logit-models-practice
Manage my subscriptions: https://tmip.org/mailinglist
Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800

Mark Bradley

I forgot to acknowledge Stephane Hess of the Leeds Choice Modeling Centre in the UK, who estimated the mixed logit models on the two ICLV studies mentioned below. It takes a lot of experience and econometric expertise to correctly specify and interpret results from complex mixed logit models, which is another factor that prevents them from being more widely used in practice.

Mark

From: Mark Bradley
Sent: Monday, June 6, 2022 12:38 PM
To: TMIP
Subject: RE: [TMIP] Mixed logit models in practice

Thanks to Peter, Juan de Dios, Dave and Greg for an interesting discussion.

Several of the activity-based models used in the US use the travel time and cost coefficient formulations developed in the report: "Improving Our Understanding of How Highway Congestion and Pricing Affect Travel Demand", TRB Strategic Highway Research Program (SHRP 2) Report S2-C04-RW-1 Download available at https://www.trb.org/Main/Blurbs/168141.aspx Peter Vovsha, Hani Mahmassani and I were PI's, and Hani estimated mixed logit models to get lognormally distributed time coefficients, similar to what Greg Erhardt described. (A 1993 TRB paper with similar models is Ben-Akiva, Bolduc and Bradley, "Estimation of Travel Choice Models with Randomly Distributed Values of Time" at https://onlinepubs.trb.org/Onlinepubs/trr/1993/1413/1413-010.pdf)

For more complex mixed logit models with several distributed parameters and (possibly) estimated correlation structures between random parameters, I would have two concerns for application.... (1) What are the run time implications of drawing from complex distributions?, and (2) When simulating over synthetic populations with millions of persons and tours, does the law of large numbers ensure that drawing just one set of parameters from the distribution(s) per choice is sufficient, or would using multiple sets of random draws per choice give more reliable results?

RSG has done a couple of applied studies in recent years using "integrated choice/latent variable" (ICLV) models estimated using mixed logit and containing several coefficients with random components and applied to large numbers of cases. One, ACRP Research Report 204 "Air Demand in a Dynamic Competitive Context with the Automobile" applied the ICLV model to all the long-distance tours generated for the full (synthesized) US population across a number of different future scenarios. (Download available at https://onlinepubs.trb.org/onlinepubs/acrp/acrp_rpt_204.pdf )

Another, NCRRP Report 4, "Intercity Passenger Rail in the Context of Dynamic Travel Markets" also applied an ICLV model in scenario analysis, but was made to be more user-accessible, applying it in Excel on a smaller sample of travelers. (https://nap.nationalacademies.org/catalog/22072/intercity-passenger-rail... )

Both of those examples are on the edge of research and practical consulting, and haven been used in practical applications elsewhere that I know of. (There are endogeneity issues in using hybrid models with latent variables in forecasting, which Caspar Chorus has written about (https://www.sciencedirect.com/science/article/abs/pii/S0967070X14001905), but that is separate from the more general question of using mixed logit models in applied model systems.

Mark

..........................................
Mark Bradley
Principal
Pronouns: he/him/his

RSG
524 Arroyo Ave| Santa Barbara, CA 93109
415.328.4766
www.rsginc.com

From: greg.erhardt=uky.edu@mg.tmip.org > On Behalf Of Greg Erhardt
Sent: Friday, June 3, 2022 7:23 AM
To: TMIP >
Subject: Re: [TMIP] Mixed logit models in practice

CAUTION - EXTERNAL EMAIL

Peter,

Mixed logit is actively used in practice in the United States and has been for over a decade. It is used specifically to estimate value of time distributions for use in agent-based models. To my knowledge, this was first done in San Francisco in about 2009 in the SF-CHAMP model. As reported in Sall et al (2010):

"The mixed-logit time-of-day model was estimated based on a stated preference survey administered to 663 travelers driving to downtown San Francisco where they were asked to trade off cost, time shifts, and mode shifts. It is applied disaggregately to all auto-person trips based on each individuals' trip purpose and value of time. Mixed logit is important in this case because it allows the user to estimate a distribution on a coefficient, rather than just the mean value. In this case, a distribution was estimated on the travel time variable, asserting a lognormal form. The cost coefficients are estimated as standard, non-distributed, coefficients segmented by income."

Standard estimation software (we used ALOGIT) can be used to estimate mixed logit models. After estimating the distribution, it is implemented using Monte Carlo simulation. Each agent draws a personal value of time from the distribution and carries that value of time with them through the simulation as one additional attribute on the agent (Erhardt et al 2008).

When applying a choice model, we simply scale the time/cost coefficients according to that agent's value of time and then apply it as a multinomial logit or nested logit model. I suspect that most users would refer to this as using a distributed value of time rather than as using mixed logit. There may be a reason not to consider this implementation strategy "true" mixed logit, and I'd be interested in understanding the implications of that if someone can shed light. Nonetheless, it is a practical way to achieve an important benefit of mixed logit-capturing user heterogeneity--for one of the most important variables in our models.

I believe this strategy is now standard practice in the activity-based models used in practice in the United States. I'm not sure that the distributions are always re-estimated, or if they're instead updated based on local wage rates. Either way, the implementation is the same. Collectively, these models have been successfully used for hundreds of planning studies. You could also do this in a trip-based model, as long as the model is agent based, such as MITO. You can demonstrate that the model will produce a different result than using an average value of time, or an average segmented by income level. This matters more when the price being tested is high because the value of time distribution tends to have a long tail-a relatively small number of people willing to pay a lot because they really don't want to be late. Maybe you're trying to catch a flight or pick up your kid before daycare closes.

References are below. If they're still available for download, I'm happy to send a copy.

Erhardt, GD, B Charlton, J Freedman, J Castiglione, and M Bradley. "Enhancement and Application of an Activity-Based Travel Model for Congestion Pricing." Presented at the Transportation Research Board Innovations in Travel Modeling Conference, Portland, Oregon, 2008.

Sall, Elizabeth, Elizabeth Bent, Jesse Koehler, Billy Charlton, and Gregory D Erhardt. "Evaluating Regional Pricing Strategies in San Francisco--Application of the SFCTA Activity-Based Regional Pricing Model." Presented at teh 89th Transportation Research Board Annual Meeting, Washington, D.C., 2010.

Kind Regards,
Greg

Greg Erhardt
Associate Professor of Civil Engineering
University of Kentucky
261 Oliver H. Raymond Bldg., Lexington, KY 40506
859-323-4856, greg.erhardt@uky.edu
transportlab.net

From: peter=transposition.com.au@mg.tmip.org On Behalf Of Peter Davidson
Sent: Tuesday, May 31, 2022 4:35 AM
To: TMIP
Subject: [TMIP] Mixed logit models in practice

CAUTION: External Sender

There's been lots of academic discussion over the last 20 years on the benefits of mixed logit models, and how they overcome the problems with the widely used hierarchical logit models. But I've been looking for good examples of the use of mixed logit models in practice. Does anyone know of any practical (city-scale or larger) transport models that make use of mixed logit? Any published information? Also would be useful to know if anyone has considered their use but decided against it, and what their reasons

were.
--
Full post: https://tmip.org/content/mixed-logit-models-practice
Manage my subscriptions: https://tmip.org/mailinglist
Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800
--
Full post: https://tmip.org/content/mixed-logit-models-practice
Manage my subscriptions: https://tmip.org/mailinglist
Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800

wigobiner

Mark and all:

While mixed logit for random coefficients on many variables at once started quite some time back from studies such as Bhat, 1998 (first released publicly in 1996 as a working paper) and Revelt and Train, 1998 (also first released in 1996 as a working paper), and was facilitated substantially by the introduction of quasi-Monte Carlo methods of simulated likelihood estimation (see Bhat, 2001 for the introduction of this technique to the field), there are new methods that generally are way more faster and computationally stable than the mixed logit-type approaches (by using a normal kernel distribution). As I wrote in a paper sometime back, "The multinomial probit (MNP) model can indeed be more parsimonious (computationally) than the mixed multinomial logit (MMNL) in many situations, such as when the number of random coefficients is much more than the number of alternatives (and when the random coefficients are normally distributed). This is because the MNP likelihood function can be expressed as an integral whose dimensionality does not depend on the number of random coefficients in the specification." Besides, we have come up with extremely efficient ways of accurately evaluating the multivariate normal cumulative distribution (MVNCD) function using analytic approximation techniques. These techniques are also much faster and accurate than traditional MSL techniques when there are quite a few normally distributed coefficients and a limited number of non-normal random coefficients. These inference techniques fall under the category of what I have labeled as the Maximum Approximate Composite Marginal Likelihood (MACML) approach. Similar techniques are available for ICLV models.

The applied field, I believe, can substantially benefit from these computationally efficient, speedier, and much more stable estimation techniques. The field has moved on from typical mixing approaches, which, while still useful, are by no means the only type of approaches that should be considered in the field for random coefficients and similar structural specifications. In fact, for most applications in the transportation field, the alternative normal-kernel approaches are far more efficient and way more practical today because of the MACML approach. Some of the comments I see about practical issues with implementing random coefficients on multiple variables may be so with traditional mixed-logit type approaches, but these do not necessarily carry over to other types of models (at least in estimation). Certainly, I would hope practical applications are not discouraged by some issues that apply only to specific types of models. Please see the following references (with links to most papers for convenience).
Bhat, C.R. (1998), "Accommodating Variations in Responsiveness to Level-of-Service Variables in Travel Mode Choice Modeling", Transportation Research Part A, Vol. 32, No. 7, pp. 495-507.
Revelt, D. and K. Train (1998). Mixed Logit with Repeated Choices: Households' Choices of Appliance Efficiency Level, The Review of Economics and Statistics (1998) 80 (4): 647-657.
Bhat, C.R. (1998), "Accommodating Flexible Substitution Patterns in Multidimensional Choice Modeling: Formulation and Application to Travel Mode and Departure Time Choice", Transportation Research Part B, Vol. 32, No. 7, pp. 455-466 (Keywords: Nested logit model, error-components logit, mixed multinomial logit, simulation estimation technique, nonwork trip modeling, travel mode choice modeling, departure time analysis). PDF version, MS Word version
Bhat, C.R. (2001), "Quasi-Random Maximum Simulated Likelihood Estimation of the Mixed Multinomial Logit Model", Transportation Research Part B, Vol. 35, No. 7, pp. 677-693.
Bhat, C.R. (2011), "The Maximum Approximate Composite Marginal Likelihood (MACML) Estimation of Multinomial Probit-Based Unordered Response Choice Models," Transportation Research Part B, Vol. 45, No. 7, pp. 923-939 (Keywords: multinomial probit, mixed models, composite marginal likelihood, discrete choice models, spatial econometrics, panel data).
Bhat, C.R., and R. Sidharthan (2011), "A Simulation Evaluation of the Maximum Approximate Composite Marginal Likelihood (MACML) Estimator for Mixed Multinomial Probit Models," Transportation Research Part B, Vol. 45, No. 7, pp. 940-953 (Keywords: mixed multinomial probit, composite marginal likelihood, maximum simulated likelihood, discrete choice models, unordered-response models, panel data). PDF version, MS Word version
Bhat, C.R., and S.K. Dubey (2014), "A New Estimation Approach to Integrate Latent Psychological Constructs in Choice Modeling," Transportation Research Part B, Vol. 67, pp. 68-85

Bhat, C.R., S.K. Dubey, and K. Nagel (2015), "Introducing Non-Normality of Latent Psychological Constructs in Choice Modeling with an Application to Bicyclist Route Choice," Transportation Research Part B, Vol. 78, pp. 341-363 (Keywords: multivariate skew-normal distribution, multinomial probit, ICLV models, MACML estimation approach, bicyclist route choice). PDF version, MS Word version

Patil, P.N., S.K. Dubey, A.R. Pinjari, E. Cherchi, R. Daziano, and C.R. Bhat (2017), "Simulation Evaluation of Emerging Estimation Techniques for Multinomial Probit Models," Journal of Choice Modelling, Vol. 23, pp. 9-20 (Keywords: discrete choice, GHK simulator, sparse grid integration, composite marginal likelihood (CML) method, MACML estimation, Bayesian Markov Chain Monte Carlo (MCMC)).

Bhat, C.R., and P.S. Lavieri (2018), "A New Mixed MNP Model Accommodating a Variety of Dependent Non-Normal Coefficient Distributions," Theory and Decision, Vol. 84, No. 2, pp. 239-275 (Keywords: copula, heterogeneity, MACML, multinomial probit, choice modeling). PDF version, MS Word version

Bhat, C.R. (2018), "New Matrix-Based Methods for the Analytic Evaluation of the Multivariate Cumulative Normal Distribution Function," Transportation Research Part B, Vol. 109, pp. 238-256

Chandra.

Chandra Bhat
University Distinguished Teaching Professor
Joe J. King Chair in Engineering
Department of Civil, Architectural and Environmental Engineering
Department of Economics (Courtesy Appointment)
The University of Texas at Austin
Austin, Texas 78712
Tel: (512) 771-9166
http://www.ce.utexas.edu/prof/bhat/home.html
Web of Science Researcher: https://publons.com/researcher/3178803/chandra-bhat/

I live and work on occupied Indigenous land and acknowledge the Carrizo & Comecrudo, Coahuiltecan, Caddo, Tonkawa, Comanche, Lipan Apache, Alabama-Coushatta, Kickapoo, Tigua Pueblo, and all the American Indian and Indigenous Peoples and communities who have been or have become a part of the lands and territories known today as Texas.

From: mark.bradley=rsginc.com@mg.tmip.org On Behalf Of Mark Bradley
Sent: Monday, June 6, 2022 3:00 PM
To: TMIP
Subject: Re: [TMIP] Mixed logit models in practice

Thanks to Peter, Juan de Dios, Dave and Greg for an interesting discussion.

Several of the activity-based models used in the US use the travel time and cost coefficient formulations developed in the report: "Improving Our Understanding of How Highway Congestion and Pricing Affect Travel Demand", TRB Strategic Highway Research Program (SHRP 2) Report S2-C04-RW-1 Download available at https://www.trb.org/Main/Blurbs/168141.aspx Peter Vovsha, Hani Mahmassani and I were PI's, and Hani estimated mixed logit models to get lognormally distributed time coefficients, similar to what Greg Erhardt described. (A 1993 TRB paper with similar models is Ben-Akiva, Bolduc and Bradley, "Estimation of Travel Choice Models with Randomly Distributed Values of Time" at https://onlinepubs.trb.org/Onlinepubs/trr/1993/1413/1413-010.pdf)

For more complex mixed logit models with several distributed parameters and (possibly) estimated correlation structures between random parameters, I would have two concerns for application.... (1) What are the run time implications of drawing from complex distributions?, and (2) When simulating over synthetic populations with millions of persons and tours, does the law of large numbers ensure that drawing just one set of parameters from the distribution(s) per choice is sufficient, or would using multiple sets of random draws per choice give more reliable results?

RSG has done a couple of applied studies in recent years using "integrated choice/latent variable" (ICLV) models estimated using mixed logit and containing several coefficients with random components and applied to large numbers of cases. One, ACRP Research Report 204 "Air Demand in a Dynamic Competitive Context with the Automobile" applied the ICLV model to all the long-distance tours generated for the full (synthesized) US population across a number of different future scenarios. (Download available at https://onlinepubs.trb.org/onlinepubs/acrp/acrp_rpt_204.pdf )

Another, NCRRP Report 4, "Intercity Passenger Rail in the Context of Dynamic Travel Markets" also applied an ICLV model in scenario analysis, but was made to be more user-accessible, applying it in Excel on a smaller sample of travelers. (https://nap.nationalacademies.org/catalog/22072/intercity-passenger-rail... )

Both of those examples are on the edge of research and practical consulting, and haven been used in practical applications elsewhere that I know of. (There are endogeneity issues in using hybrid models with latent variables in forecasting, which Caspar Chorus has written about (https://www.sciencedirect.com/science/article/abs/pii/S0967070X14001905), but that is separate from the more general question of using mixed logit models in applied model systems.

Mark

..........................................
Mark Bradley
Principal
Pronouns: he/him/his

RSG
524 Arroyo Ave| Santa Barbara, CA 93109
415.328.4766
www.rsginc.com

From: greg.erhardt=uky.edu@mg.tmip.org On Behalf Of Greg Erhardt
Sent: Friday, June 3, 2022 7:23 AM
To: TMIP
Subject: Re: [TMIP] Mixed logit models in practice

CAUTION - EXTERNAL EMAIL

Peter,

Mixed logit is actively used in practice in the United States and has been for over a decade. It is used specifically to estimate value of time distributions for use in agent-based models. To my knowledge, this was first done in San Francisco in about 2009 in the SF-CHAMP model. As reported in Sall et al (2010):

"The mixed-logit time-of-day model was estimated based on a stated preference survey administered to 663 travelers driving to downtown San Francisco where they were asked to trade off cost, time shifts, and mode shifts. It is applied disaggregately to all auto-person trips based on each individuals' trip purpose and value of time. Mixed logit is important in this case because it allows the user to estimate a distribution on a coefficient, rather than just the mean value. In this case, a distribution was estimated on the travel time variable, asserting a lognormal form. The cost coefficients are estimated as standard, non-distributed, coefficients segmented by income."

Standard estimation software (we used ALOGIT) can be used to estimate mixed logit models. After estimating the distribution, it is implemented using Monte Carlo simulation. Each agent draws a personal value of time from the distribution and carries that value of time with them through the simulation as one additional attribute on the agent (Erhardt et al 2008).

When applying a choice model, we simply scale the time/cost coefficients according to that agent's value of time and then apply it as a multinomial logit or nested logit model. I suspect that most users would refer to this as using a distributed value of time rather than as using mixed logit. There may be a reason not to consider this implementation strategy "true" mixed logit, and I'd be interested in understanding the implications of that if someone can shed light. Nonetheless, it is a practical way to achieve an important benefit of mixed logit-capturing user heterogeneity--for one of the most important variables in our models.

I believe this strategy is now standard practice in the activity-based models used in practice in the United States. I'm not sure that the distributions are always re-estimated, or if they're instead updated based on local wage rates. Either way, the implementation is the same. Collectively, these models have been successfully used for hundreds of planning studies. You could also do this in a trip-based model, as long as the model is agent based, such as MITO. You can demonstrate that the model will produce a different result than using an average value of time, or an average segmented by income level. This matters more when the price being tested is high because the value of time distribution tends to have a long tail-a relatively small number of people willing to pay a lot because they really don't want to be late. Maybe you're trying to catch a flight or pick up your kid before daycare closes.

References are below. If they're still available for download, I'm happy to send a copy.

Erhardt, GD, B Charlton, J Freedman, J Castiglione, and M Bradley. "Enhancement and Application of an Activity-Based Travel Model for Congestion Pricing." Presented at the Transportation Research Board Innovations in Travel Modeling Conference, Portland, Oregon, 2008.

Sall, Elizabeth, Elizabeth Bent, Jesse Koehler, Billy Charlton, and Gregory D Erhardt. "Evaluating Regional Pricing Strategies in San Francisco--Application of the SFCTA Activity-Based Regional Pricing Model." Presented at teh 89th Transportation Research Board Annual Meeting, Washington, D.C., 2010.

Kind Regards,
Greg

Greg Erhardt
Associate Professor of Civil Engineering
University of Kentucky
261 Oliver H. Raymond Bldg., Lexington, KY 40506
859-323-4856, greg.erhardt@uky.edu
transportlab.net

From: peter=transposition.com.au@mg.tmip.org On Behalf Of Peter Davidson
Sent: Tuesday, May 31, 2022 4:35 AM
To: TMIP
Subject: [TMIP] Mixed logit models in practice

CAUTION: External Sender

There's been lots of academic discussion over the last 20 years on the benefits of mixed logit models, and how they overcome the problems with the widely used hierarchical logit models. But I've been looking for good examples of the use of mixed logit models in practice. Does anyone know of any practical (city-scale or larger) transport models that make use of mixed logit? Any published information? Also would be useful to know if anyone has considered their use but decided against it, and what their reasons

were.
--
Full post: https://tmip.org/content/mixed-logit-models-practice
Manage my subscriptions: https://tmip.org/mailinglist
Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800
--
Full post: https://tmip.org/content/mixed-logit-models-practice
Manage my subscriptions: https://tmip.org/mailinglist
Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800
--
Full post: https://tmip.org/content/mixed-logit-models-practice
Manage my subscriptions: https://tmip.org/mailinglist
Stop emails for this post: https://tmip.org/mailinglist/unsubscribe/13800