[elm-discuss] Debouncing, throttling, exponential backoff, relating effects to time - valid use of effect manager?

[Henry]

Evan has said before that there are around 10 valid uses of an effect
manager, do you think relating effects to time is one of them?

[Simon]

personally, yes. You can do debouncing on a case by case basis without an
effects manager, but a generic one seems to require it

Post by Henry
Evan has said before that there are around 10 valid uses of an effect
manager, do you think relating effects to time is one of them?

[Ryan Rempel]

Post by Henry
Evan has said before that there are around 10 valid uses of an effect
manager, do you think relating effects to time is one of them?

One way of thinking about this question is to compare existing debouncers
that use an effects manager vs. those which do not.

Two debouncers which use an effects manager are:

https://github.com/unbounce/elm-debounce
https://github.com/mceldeen/elm-debouncer

Two debouncers which do not use an effects manager are:

http://package.elm-lang.org/packages/jinjor/elm-debounce/latest
http://package.elm-lang.org/packages/mpizenberg/elm-debounce/latest

Now, both approaches need to keep some state, of course.

The advantage of an effects manager is that it can keep some state behind
the scenes, so to speak, without requiring the user of the package to
integrate the state into the `model`, `msg` and `update` scheme in the
usual way.

However, I don't think there is anything an effects manager can accomplish
(at least with respect to debouncing) that cannot also be done without an
effects manager, at the cost of additional wiring and verbosity. That is,
at the cost of making the state visible, and requiring you to integrate it
into your `model`, `msg` and `update` scheme, you can achieve everything
you'd want with respect to debouncing without using an effects module.

In fact, there are some advantages in not using an effects module. If you
look at the debouncers that use an effects module, they employ a string ID
in order to distinguish between one debouncer and another. (That is, the
module is possibly tracking the state for multiple debouncers internally,
so you need to provide a string ID to distinguish between one and another).
But, of course, this isn't entirely satisfactory, since it forces you to
maintain some scheme of globally-unique strings within your program. (Which
is not necessarily that hard, really, but it is something which you'd
normally want to avoid).

That problem doesn't arise if you have to explicitly integrate some state
into your `model`, `msg` and `update` in the usual way, since then you just
provide the relevant state when needed ... you can't accidentally refer to
the wrong bit of state by providing a string ID that is also used elsewhere.

Of course, there may be a clever way to write an effects module that avoids
this problem.

[Henry]

Thank you Ryan for the excellent reply! You made a lot of good points, and
I appreciate the overview on the available libraries and their drawbacks.

I feel like the effect manager conveys the intent of the code better, you
get to say "I want this to happen" and then shove the state in a lock box,
throw it in a closet, throw the closet in a river, and pretend like the
hole in the side of your house was always there, and is perfectly normal.
The use of global string IDs is a big downside though, it feels dirty
writing code like that when everything else is wrapped in ribbons, laced
with foil, and covered in the wondrous glitter of the type system.

With relating effects to time, the main examples I think of are: debouncing
searches, throttling clicks/events (you can only do X once every 10
seconds), exponential backoff (for network requests). I've asked a few
people if they know the name of that class of problems, relating events to
time, and I still don't know the name of them, but some friends now think
I'm an idiot who doesn't know what "history" is.

I was trying to think of various ways it could look in the language (with
little regard to what is feasible...), and these are a few

let
debouncedInput = debounce 200 onInputininput [ debouncedInput SearchOnServer ]


This way the message and model are unchanged, and debouncedInput handles
everything auto-magically, much the same way onInput does already. Ideally
you could use that with Html.Events.on, or any of its ilk. The upside here
is that it is very clear what is happening, because you get to state it
right where it happens, but it would need to be implemented in VirtualDom
and it would only work for HTML events. It could be use for throttling
chat messages, button clicks, scroll events, etc. The event listener
implementation is in VirtualDom here
(https://github.com/elm-lang/virtual-dom/blob/dev/src/Elm/Kernel/VirtualDom.js#L521-L527)

Another potential way (ala unbounce or mceldeen's implementations) is
handling the individual message, but debouncing a subsequent message. We
can collect the text from the search bar, and run the search after the user
has quit typing.

update : Msg -> Model -> ( Model, Cmd Msg )update msg model =
case msg of
SearchUpdate newSearch ->
( { model | search = newSearch }, debounce "search" 200 PerformSearch )


I feel it would be even better to be able to write as

( { model | search = newSearch }, debounce 200 PerformSearch )


For HTTP requests I would use the "second message' pattern as well, with
one event sending a Cmd that will resolve later and then make the request.
With exponential backoff there is added state for the number of tries so
far, so my ideal API handles all of that for me with magic

Http.getWithExponentialBackoffAndAMakeAMartini decoder ("https://grass-fed-lemonade.com/search" ++ query)


But a more realistic version is probably something like

update : Msg -> Model -> ( Model, Cmd Msg )update msg model =
case msg of
SearchFailed err ->
( { model | failed = model.failed + 1 }
, Process.sleep (model.failed * 2 * 1000)
|> Task.perform (\_ -> PerformSearch)


I think any inclusion of "magic" (re: hidden state), should be weighed
carefully, and in this instance I think the increased clarity of the code
is worth it, though clarity is subjective as well.

Post by Ryan Rempel

Post by Henry
Evan has said before that there are around 10 valid uses of an effect
manager, do you think relating effects to time is one of them?

One way of thinking about this question is to compare existing debouncers
that use an effects manager vs. those which do not.
https://github.com/unbounce/elm-debounce
https://github.com/mceldeen/elm-debouncer
http://package.elm-lang.org/packages/jinjor/elm-debounce/latest
http://package.elm-lang.org/packages/mpizenberg/elm-debounce/latest
Now, both approaches need to keep some state, of course.
The advantage of an effects manager is that it can keep some state behind
the scenes, so to speak, without requiring the user of the package to
integrate the state into the `model`, `msg` and `update` scheme in the
usual way.
However, I don't think there is anything an effects manager can accomplish
(at least with respect to debouncing) that cannot also be done without an
effects manager, at the cost of additional wiring and verbosity. That is,
at the cost of making the state visible, and requiring you to integrate it
into your `model`, `msg` and `update` scheme, you can achieve everything
you'd want with respect to debouncing without using an effects module.
In fact, there are some advantages in not using an effects module. If you
look at the debouncers that use an effects module, they employ a string ID
in order to distinguish between one debouncer and another. (That is, the
module is possibly tracking the state for multiple debouncers internally,
so you need to provide a string ID to distinguish between one and another).
But, of course, this isn't entirely satisfactory, since it forces you to
maintain some scheme of globally-unique strings within your program. (Which
is not necessarily that hard, really, but it is something which you'd
normally want to avoid).
That problem doesn't arise if you have to explicitly integrate some state
into your `model`, `msg` and `update` in the usual way, since then you just
provide the relevant state when needed ... you can't accidentally refer to
the wrong bit of state by providing a string ID that is also used elsewhere.
Of course, there may be a clever way to write an effects module that
avoids this problem.

[Ilias Van Peer]

Your final example (exponential backoff on HTTP requests) doesn't need any
magic.

You can use `Http.toTask` to turn it into a task and implement a generic
"retry task at most x times with backoff" like this:

retry : Int -> Time -> Task x a -> Task x a
retry maxTries backOff task =
if maxTries == 0 then
task
else
Task.onError
(\_ ->
Process.sleep backOff
|> Task.andThen
(\_ -> retry (maxTries - 1) (backOff * 2) task)
)
task

Post by Henry
Thank you Ryan for the excellent reply! You made a lot of good points, and
I appreciate the overview on the available libraries and their drawbacks.
I feel like the effect manager conveys the intent of the code better, you
get to say "I want this to happen" and then shove the state in a lock box,
throw it in a closet, throw the closet in a river, and pretend like the
hole in the side of your house was always there, and is perfectly normal.
The use of global string IDs is a big downside though, it feels dirty
writing code like that when everything else is wrapped in ribbons, laced
with foil, and covered in the wondrous glitter of the type system.
debouncing searches, throttling clicks/events (you can only do X once every
10 seconds), exponential backoff (for network requests). I've asked a few
people if they know the name of that class of problems, relating events to
time, and I still don't know the name of them, but some friends now think
I'm an idiot who doesn't know what "history" is.
I was trying to think of various ways it could look in the language (with
little regard to what is feasible...), and these are a few
let
debouncedInput = debounce 200 onInputininput [ debouncedInput SearchOnServer ]
This way the message and model are unchanged, and debouncedInput handles
everything auto-magically, much the same way onInput does already.
Ideally you could use that with Html.Events.on, or any of its ilk. The
upside here is that it is very clear what is happening, because you get to
state it right where it happens, but it would need to be implemented in
VirtualDom and it would only work for HTML events. It could be use for
throttling chat messages, button clicks, scroll events, etc. The event
listener implementation is in VirtualDom here (
https://github.com/elm-lang/virtual-dom/blob/dev/src/Elm/Kernel/VirtualDom.js#L521-L527
)
Another potential way (ala unbounce or mceldeen's implementations) is
handling the individual message, but debouncing a subsequent message. We
can collect the text from the search bar, and run the search after the user
has quit typing.
update : Msg -> Model -> ( Model, Cmd Msg )update msg model =
case msg of
SearchUpdate newSearch ->
( { model | search = newSearch }, debounce "search" 200 PerformSearch )
I feel it would be even better to be able to write as
( { model | search = newSearch }, debounce 200 PerformSearch )
For HTTP requests I would use the "second message' pattern as well, with
one event sending a Cmd that will resolve later and then make the
request. With exponential backoff there is added state for the number of
tries so far, so my ideal API handles all of that for me with magic
Http.getWithExponentialBackoffAndAMakeAMartini decoder ("https://grass-fed-lemonade.com/search" ++ query)
But a more realistic version is probably something like
update : Msg -> Model -> ( Model, Cmd Msg )update msg model =
case msg of
SearchFailed err ->
( { model | failed = model.failed + 1 }
, Process.sleep (model.failed * 2 * 1000)
|> Task.perform (\_ -> PerformSearch)
I think any inclusion of "magic" (re: hidden state), should be weighed
carefully, and in this instance I think the increased clarity of the code
is worth it, though clarity is subjective as well.

Post by Ryan Rempel

Post by Henry
Evan has said before that there are around 10 valid uses of an effect
manager, do you think relating effects to time is one of them?

One way of thinking about this question is to compare existing debouncers
that use an effects manager vs. those which do not.
https://github.com/unbounce/elm-debounce
https://github.com/mceldeen/elm-debouncer
http://package.elm-lang.org/packages/jinjor/elm-debounce/latest
http://package.elm-lang.org/packages/mpizenberg/elm-debounce/latest
Now, both approaches need to keep some state, of course.
The advantage of an effects manager is that it can keep some state behind
the scenes, so to speak, without requiring the user of the package to
integrate the state into the `model`, `msg` and `update` scheme in the
usual way.
However, I don't think there is anything an effects manager can
accomplish (at least with respect to debouncing) that cannot also be done
without an effects manager, at the cost of additional wiring and verbosity.
That is, at the cost of making the state visible, and requiring you to
integrate it into your `model`, `msg` and `update` scheme, you can achieve
everything you'd want with respect to debouncing without using an effects
module.
In fact, there are some advantages in not using an effects module. If you
look at the debouncers that use an effects module, they employ a string ID
in order to distinguish between one debouncer and another. (That is, the
module is possibly tracking the state for multiple debouncers internally,
so you need to provide a string ID to distinguish between one and another).
But, of course, this isn't entirely satisfactory, since it forces you to
maintain some scheme of globally-unique strings within your program. (Which
is not necessarily that hard, really, but it is something which you'd
normally want to avoid).
That problem doesn't arise if you have to explicitly integrate some state
into your `model`, `msg` and `update` in the usual way, since then you just
provide the relevant state when needed ... you can't accidentally refer to
the wrong bit of state by providing a string ID that is also used elsewhere.
Of course, there may be a clever way to write an effects module that
avoids this problem.