Ordered Effects

[mike-thompson-day8]

If I had my time over again, I would make reg-event-fx return a seq of effects, rather than a map of effects. The resulting effects DSL would be more expressive, at the cost of some nesting.

I have previously teased out the issues a bit here:
https://gist.github.com/mike-thompson-day8/1388e68391f4c6c1373c9555c08c7114#using-data-litterals

Advantages:

1. It gives a way to order effects (not that useful, but an interesting property)
2. no need for dispatch-n , just use :dispatch multiple times (this same -n issues comes up a lot with other effects like http GETs etc)

Disadvantages:

3. slightly noisier, visually, with the additional structural nesting
4. is a breaking change

It might be possible to work around the breaking change problem by introducing a new reg-event-fx2 which expects the alternatively structured effects return. Allowing us to leave the old reg-event-fx alone. But it isn't all sunshine and roses - we'd end up with interceptors which worked with one kind of effects, but not the other. So then there would be v1 interceptors and v2 interceptors. One worked with one kind of effect but not the other.

@olivergeorge has made an interesting suggestion which fixes the "breaking change" side of things, at the expense of a tiny bit of boilerplate. He suggests introducing a new :fx effect for use with reg-event-fx which can take seq of other effects.

(reg-event-fx 
   :token 
   (fn [{:keys [db]} [_ id]]
      {:db    (assoc db  :something id)
       :fx    [{:dispatch [:blah]}                             ;; <-- :fx takes a seq of effects
               {:dispatch [:other]}
               {:http  {:url   "http://abc.com/endpont  :method "GET"}}]}))

Notice the new :fx effect for which a seq of other effects is provided.

Question: if done this way, what is the ordering between :db and those in :fx? :db first? Probably.

Question: I regard :db to be a normal effect (it is just a side effect on app-db), so it could be put into :fx too. (What happens if it is not, what is the ordering?)

(reg-event-fx 
   :token 
   (fn [{:keys [db]} [_ id]]
      {:fx  [{:db    (assoc db  :something id)}
             {:dispatch [:blah]}
             {:dispatch [:other]}
             {:http  {:url   "http://abc.com/endpont  :method "GET"}}]}))

Question: maybe to reduce noise, just pairs like this

(reg-event-fx 
   :token 
   (fn [{:keys [db]} [_ id]]
      {:fx  [:db      (assoc db  :something id)   ;; <-- not a map, just a pair
             :dispatch [:blah]
             :dispatch [:other]
             :http     {:url   "http://abc.com/endpont  :method "GET"}]}))

At the moment, my inclination is this arrangement (which is slightly hybrid):

(reg-event-fx 
   :token 
   (fn [{:keys [db]} [_ id]]
      {:db  (assoc db  :something id)   ;; <-- keep :db seperate
       :fx  [:dispatch [:blah]                  ;; <-- use pairs
             :dispatch [:other]
             :http     {:url   "http://abc.com/endpont  :method "GET"}]}))

Conditional Exclusion

(reg-event-fx 
   :token 
   (fn [{:keys [db]} [_ id]]
      {:db  (assoc db  :something id)  
       :fx  [{:dispatch [:blah]}
             (when cond? {:dispatch [:other]})       ;; <-- conditionally exclude this effect.  nils ignored? 
             {:http     {:url   "http://abc.com/endpont  :method "GET"}}]}))

Or if pairs are used ....

(reg-event-fx 
   :token 
   (fn [{:keys [db]} [_ id]]
      {:db  (assoc db  :something id)  
       :fx  [:dispatch [:blah]                 
             :dispatch  (when cond? [:other])       ;; <-- conditionally exclude this effect.  nils ignored? 
             :http     {:url   "http://abc.com/endpont  :method "GET"}]}))

As you can see, If paris are used, then It isn't neat to conditionally omit both. Only the value part of the pair could be made nil. This could be a bit clumsy because certain effects actually have a value nil value. Eg; :go-fullscreen

[olivergeorge]

Thanks for putting this together.

I vote this :fx form

(reg-event-fx 
   :token 
   (fn [{:keys [db]} [_ id]]
      {:db    (assoc db  :something id)
       :fx    [{:dispatch [:blah]}                             ;; <-- :fx takes a seq of effects
               {:dispatch [:other]}
               {:http  {:url   "http://abc.com/endpont  :method "GET"}}]}))

My argument being

• :db is special. Its meaning is old-state (as arg) and new-state (as ret value). (To my mind, that it's implemented via reg-fx is really an implementation detail)
• it's symmetrical (arg -> ret value) allowing for composition via comp or ->.
• cofx can flow through composed handlers.

To my mind, facilitating handler composition idea is a huge unlock for re-frame. The current handler calling convention complicates reuse.

For example (below) but this is taking it to a logical extreme which might not be justified with such simple handlers.

(defn set-something [m]
  (assoc-in m [:db :something] (get-in m [:event 1])))

(defn fetch-data [m]
  (update-in m [:fx] conj {:http {:url "http://abc.com/endpont" :method "GET"}}))

(defn dispatch-other [m]
  (update-in m [:fx] conj {:dispatch [:other]}))

(reg-event-fx2 :token (comp set-something fetch-data dispatch-other))

PS. This discussion provides some insight into the nature of :db. It thinks its useful to recognise it's domain use (state) without getting distracted by the implementation details (cofx + fx).

PPS. I'd prefer :fx to expect [{:bah 1} ...] over [:bah 1 ...] for clarity/composability reasons. It's probably a matter of taste but my sense is the CLJ world prefers maps with semantic keys over vectors & conventions.

PPPS. I think :fx could be made to simplify the :gofullscreen case (fx with no args) if it's a sticking point. Similarly, dispatch style vectors instead of maps would be compatible and possibly desirable to avoid boilerplate in fx associated with unpacking args. Perhaps the spec for :fx would become...

(s/coll (s/or :noop nil?
              :fx-0 keyword?
              :fx-m map?
              :fx-v (s/cat :id keyword? :args (s/* any?))))

PPPPS. Perhaps :db should be registered as a new type of thing allowing for different state models. State being something which passes through handlers. So it's "cofx like" in providing a value to handler and "fx like" in changing app-db on output but specifically associated with one key (e.g. :db) and by convention is passed through by handlers. That seems like a nice way to allow pluggable state management.

(reg-state :db 
  (fn [] @app-db) 
  (fn [db] (reset! app-db db)))

[danielytics]

Regarding db being special, would the following be valid? (Using the "pairs" syntax, although olivergeorge makes a good case for alternative syntax)

(reg-event-fx 
   :token 
   (fn [{:keys [db]} [_ id]]
      {:db  (assoc db  :something id)
       :dispatch  [:foo]              ;; <-- using an "old-style" effect
       :fx  [:dispatch [:blah]
             :dispatch [:other]
             :http     {:url   "http://abc.com/endpont  :method "GET"}]}))

If yes (it is valid), then DB isn't really treated as special and it should probably be ok to have multiple db effects in the fx vector too. In this case, its important to define the order (I would say, all old-style non-:fx effects first, in arbitrary order, followed by the new-style :fx effects, in the specified order).

If no (it is not valid), then its not really backward compatible at all, in which case I'd suggest just dropping the outer map and returning the fx vector of pairs directly.

[olivergeorge]

Using a (ref-fx :fx ...) would be backward compatible (technically) and :db, like :dispatch, could work in both places. Problem is that if :db appears twice it would cause a conflict which clobbers one of the changes.

I guess a fix for that might be to keep a list of effects which can appear only once and exclude them from consideration in the (ref-fx :fx ...) form.

[mike-thompson-day8]

I've let this idea marinate for a few of days and I'm yet to see any downsides. In fact, I'm getting more sure this is a great idea. Even the key :fx works in nicely with reg-fx etc.

I will continue to let it sit for another couple of days just to see if any gremlins or improvements come to mind, before making it happen. While it is only about 5 lines of code, it is all the docs changes which I'm not looking forward to - adding this feature will lead to a change to best practice. I'll have to take on that task incrementally.

The benefits:

1. no breaking changes
2. ordered effects
3. the ability to automatically handle the *-n variant of all effects (eg: no need for :dispatch-n)
4. a more natural way to compose event handlers from smaller functions.

I'm currently favour this form:

(reg-event-fx 
   :token 
  (fn [{:keys [db]} [_ id]]
    {:db (assoc db :something id)
     :fx [{:dispatch  [:blah]}   
          (when seventies? {:hair "side burns"})     ;; nils are ignored
          {:http    {:method :GET  :url "http:/abc.com/endpoint"}}]}))

Notes:

• the key :fx will be associated with a seq, typically a vector
• the effects in :fx will be processed in order
• each element in the seq will be a map which represents one effect. Eg {:dispatch [:blah]}
• if an element of :fx is nil, it will be ignored, which allows effects to be conditionally included via a when
• visually, this is more "noisy" than using the "pairs" format, and aesthetics do matter, but as you point out, this approach is easier to model mentally (teach and get right)
• The :db effect will be processed before those in :fx. In fact, :db will now be processed before all other effects, and :fx is just one of them.
• produce a warning if the :fx seq contains any :db effects. Just a warning. Some way to turn the warning off?

Point 4 - Composing Event Handler

Point 4 of the benefits list (above) is the most intriguing, and the main reason I'm still dwelling on it. If we do this, I want to maximise the value we get from this approach. Now we have this idea in our head, and the smell of it in our nostrils, are there any other tweaks which would add even more value?

At a minimum, composing an event handler from smaller functions is now a lot more natural. For example:

(reg-event-fx 
   :token 
  (fn [{:keys [db]} [_ id]]
   (-> {:db db 
        :fx  [{:dispatch [:token2]}]}                 ;; base structure
      (start-ui-throber)        ;; <-- might assoc into `:db`
      (GET "http://abc.com/endpoint")      ;; adds to `:fx`
      (POST "http://blah.com/endpoint")))  ;; adds to `:fx`

Notes:

• in this imaginary example …
• the event handler starts by creating a data structure {:db db :fx []} which it then threads through the other functions, composing the effects. Except that isn't entirely true, because it adds a :dispatch effect using data literals at the time the structure is created.
• GET and POST are imaginary functions which will conj an effect onto the :fx part of the supplied argument
• start-ui-throbber will probably assoc a value into some path of :db to cause the twirly thing to appear for the user

Compare the code above with the existing approach below which tends to encourage "effects via data literals".

(reg-event-fx 
   :token 
  (fn [{:keys [db]} [_ id]]
    {:db (assoc db :twirly true) 
     :dispatch [:blah]
     :http  (list 
                  {:method :GET :url "http://abc.com/endpoint" ...}
                  {:method :POST :url "http://blah.com/endpoint" ...})}))

There is something pleasingly explicit to me about maximising the use of data literals, but composing effects via functions is also an interesting approach, and I like the choice, and I like being able to mix the approaches as I see fit.

FSM Example

I am particularly happy because this new approach will make it easier to handle Finite State machines (FSM) within re-frame. I look forward to being able to write:

(reg-event-fx 
   :token 
  (fn [{:keys [db]} event]
   (-> {:db db :fx []}                 ;; base structure
       (fsm fsm-definition event)))    ;; process the finite state machine

where:

• fsm is some general execution engine (function) which will step a definition forward.
• the new :db and :fx is just a lot more composable in the case that the FSM has to step through a couple of states before going async again.

Why :fx and :db

I still regard :db is just another side effect - it just involves a reset! of app-db.

But, we all intuit that :db is a different kind of side effect to those in :fx.

But why? I'd enjoy getting a crisper explanation for this.

It is something to do with :fx effects being independent of each other and able to be done "later". But the:db side effect need to accumulate in situ because the accumulated results might be used by the next function.

reg-event-fx2

@olivergeorge I'm unclear on your reg-event-fx2 proposal (duplicated below)

(reg-event-fx2 :token (comp set-something fetch-data dispatch-other))

Are you suggesting a new registration function which sets up the {:db db :fx []} part automatically and then threads that through the supplied event handler (in your example created via comp?

[p-himik]

adding this feature will lead to a change to best practice

But why? If someone doesn't care about the order (I find it to be the case in 90% of the situations I encounter), why suddenly writing

{:db (assoc db :a 1)
 :fx [{:dispatch [:do-stuff-to-a]}
      {:http {:method :get, :on-success [:inc-a], :on-failure [:dec-a]}}]}

is preferred to a simpler and yet as sufficient

{:db (assoc db :a 1)
 :dispatch [:do-stuff-to-a]
 :http {:method :get, :on-success [:inc-a], :on-failure [:dec-a]}}

[mike-thompson-day8]

@p-himik
This really isn't about ordered effects. (Which is confusing, given the issue's title, sorry :-)). As you say, that's a minor usecase.

For one reason, it removes the need for all the *-n variants: currently there is both :dispatch and dispatch-n. The same exists for http effects. It ends up happening for most effects. (For backwards compatibility, these will hang around, but they will no longer be featured in the docs).

Also, it allows for easier composing of effectfulness-causing functions. This is the "Point 4" I talked about above. If you have one function which wants to cause a dispatch, should it be adding to :dispatch or :dispatch-n ?? In the new scheme, always just conj onto :fx

It will become "best practice" because it just gently solves a few sticking points (no one of them a show stopper). And it means it is more natural to "compose effects" (Point 4). And because it is better if there is only one recommended way to do things.

[p-himik]

it is better if there is only one recommended way to do things

Indeed. But I still don't really get why it has to be the :fx method.
And to be honest, I don't really get why there has to be a "recommended" way. I like having a set of tools, and I like to be able to choose the tool according to the task at hand and not because it was labeled as "recommended". We have at least 3 ways to get the first element of a vector that I can think of right away, and none of them is labeled as recommended.

Yes, :fx solves two pain points which one can encounter maybe once in 10-20 instances of using ref-event-* (just to mention it again - I can only speak from my own practice. Maybe there are people for whom the number is 3 or 100).
And I wouldn't whine were it the only thing that it does. Alas, :fx also introduces a trade-off - it makes things much more verbose in arguably one of the most frequent use cases, :dispatch-n:

{:dispatch-n [[:display-pop-up]
              [:start-throbber]
              [:fetch-data]
              [:start-countdown]]}
;; vs.
{:fx [{:dispatch [:display-pop-up]}
      {:dispatch [:start-throbber]}
      {:dispatch [:fetch-data]}
      {:dispatch [:start-countdown]}}

I definitely see such scenario much more often than the need to compose or order the fx. And I personally would never recommend using :fx in this case if you don't know for sure that you will need to add some other effects in the middle of those events.

Actually, writing the above gave me an idea. What if :fx accepts a seq of not only maps but also of event vectors? In that case, the above could be written as

{:fx [[:display-pop-up]
      [:start-throbber]
      [:fetch-data]
      [:start-countdown]]}

It as all the benefits of the initially proposed version of :fx but it's also even less verbose (by the whopping 8 characters, yes) than :dispatch-n. I admit, it feels a bit "dirty". But at the same time it seems practical enough to justify that. Especially given that having either a vector or a map is somewhat a common practice with effects in general, e.g. as it's the case with :http-xhrio.

[mike-thompson-day8]

@p-himik

I don't really get why there has to be a "recommended" way.

The docs have to adopt a style. Best for newbies that way.

But I still don't really get why it has to be the :fx method.

I detailed the four benefits above. I also pointed out that, to me, the most interesting was "Point 4".

I'm confident that it will make it easier to implement certain aspects of FSM. For example, I can see my way clear to finishing this library: https://github.com/day8/re-frame-http-fx-alpha (which stalled because I could never work out a clean way to compose state handlers)

But, hey, it is backwards compatible, so no need to change your existing.

[ikitommi]

This must be evil, but would be possible:

(re-frame/reg-event-fx ::fx
  (fn [{:keys [db]} [_ fx]] (fx {:db db :fx []})))

(re-frame/dispatch [::fx (comp set-something fetch-data dispatch-other)])

[kirillsalykin]

What do you think about using vectors - so having the same structure as used in subs, I think it feels more consistent.

Something like this:

{:fx [[:dispatch [:display-pop-up]]
      [:dispatch [:start-throbber]]
      [:dispatch [:fetch-data]]
      [:dispatch [:start-countdown]]}

[mike-thompson-day8]

@ikitommi
Evil thoughts can be very instructive and helpful :-)
I prefer my events to be pure data, but it is interesting and instructive that your sketch is possible.

@kirillsalykin
How would you propose to do an effect like this one: https://github.com/day8/re-frame-http-fx

[mike-thompson-day8]

Actions:

• add the new :fx event handler
  • add new reg-fx for :fx (warnings if :db effects found within :fx seq)
  • add to API docs
• add guarantee that :db effect will be done first
  • at the code level
  • update tutorial docs in this area (there is a comment about no ordering somewhere, so remove that)
  • update API docs to specify that :db is done first
• update Changes/Releases docs
• release a new version
• review tutorial docs overall. At what point do we introduce :fx?

We'll defer introducing any new reg-event-fx2 until we have more experience using this feature.

[kirillsalykin]

@mike-thompson-day8

I imagine it like this:

{:fx [[:http-xhrio {:method          :post
                             :uri             "https://httpbin.org/post"
                             :params          data
                             :timeout         5000
                             :format          (ajax/json-request-format)
                             :response-format (ajax/json-response-format {:keywords? true})
                             :on-success      [::success-post-result]
                             :on-failure      [::failure-post-result]]}

Basic idea is to use vector literal instead of map literal - so it has same syntax as reg-sub
Not something very important, just for consistency.

[kirillsalykin]

@mike-thompson-day8

If I had my time over again, I would make reg-event-fx return a seq of effects, rather than a map of effects. The resulting effects DSL would be more expressive, at the cost of some nesting.

Would it be possible to differentiate old behaviour from new based on type then?
eg if reg-event-fx returns map - it is old way fx, if vector/list - then the new way?

Thanks

[mike-thompson-day8]

@kirillsalykin

Would it be possible to differentiate old behaviour from new based on type then?

Yes, it would be possible to differentiate - if you knew there were two kinds of effects. But existing interceptors expect a map of effects (they don't even know to try and differentiate). They would break.

Basic idea is to use vector literal instead of map literal - so it has same syntax as reg-sub

What you suggest is possible, but I don't see any advantages. At this point, will be going with maps.