Asynchronous command chaining

[graphex]

Looking at the macroCommand class, I'm wondering what the recommended practice is for chaining asynchronous commands, as macroCommand appears to just call the next command once the execute method returns, which may not actually be the end of the command.

When I'm getting something from the server, I'm typically going to have a callback which will need to execute before I want the next command to fire. I'd rather not couple all asynchronous commands with a specific command to execute next, as this often changes depending on the context.

I believe that Cairngorm's SequenceCommand handles this pretty well, and I wondered if anyone had any thoughts on implementing something like that in PureMVC. I'm guessing that using SimpleCommand with a payload that includes the command to call next would work, but I did like the simplicity of calling executeNextCommand() when my asynchronous command completed, and I like aggregating my FIFO command stack in a 'meta command' rather than just running it through a notification payload.

I'd try overriding the macroCommand's execute function, but it is declared final for some reason. Is there a way to easily override macroCommand to account for asynchronous subcommands, or is that execute function final for a reason?

Sean

[puremvc]

The short answer is Commands aren't the ones doing service calls in PureMVC, its the Proxies doing that. Commands are generally executing synchronous business logic within the client.When a Proxy hears back from a service it will generally send a Notification you can respond to with a Command.

The execute method of MacroCommand is final to prevent its  being inadvertently overridden in a subclass, as you would with a SimpleCommand.

[graphex]

I understand how an asynchronous chaining command might not make it in to the framework, and I see why the execute command is final, but it would be nice if there were a way to encapsulate the sequence within the command.

In my own application, I did just have my proxy send out another notification. However, sometimes it is certainly nice to have a centralized command chain without having to chase a whole trail of asynchronous proxy responses, or put in a bunch of transient "what I'm supposed to do after the webservice call returns" variables in the model. Commands are designed to be where big things involving lots of application-wide stuff can be centralized, why limit them to being synchronous things?

I think a good example is 'lazy authentication'. A user deep links to a part of the application where authentication is necessary. A command is issued to 1)check authentication via webservice, 2)modify the model to update the view to complete their request. If the webservice call in #1 fails, standard authorization comes up, eventually calling nextCommand() once a successful authorization takes place. The authorization component only has to know to call nextCommand(), and the model doesn't have to have tons of transient state flags and logic to store what was supposed to happen next.

I'd say the existence of MacroCommand essentially justifies the general pattern, but in real world use, the subcommands of a MacroCommand class can (and frequently should) depend upon data retrieved asynchronously. So, any chance a SequenceCommand pattern could be introduced to basically do the nice architectural things for asynchronous subcommands that a MacroCommand does for synchronous subcommands?

[puremvc]

You can handle this with the AsyncToken pattern.

Whenever you make a service call in Flex, you get back an AsyncToken object. This is a dynamic class that you can add properties to. When the service returns, this token will be available on the ResultEvent or FaultEvent as the property event.token.

So the Proxy (or Delegate) sets a property on the Token when the call is made. When the service returns, the Proxy sends the notification name that is stored in the token.

Instead of calling nextCommand() you set a property on the Proxy, which is an implicit setter, with the name of the notification to send if the user is authenticated. If the authentication hasn't happened yet, then it takes place, and the continuation of the use case is accomplished by sending the notification that was specified to the proxy earlier.

This methodology is more compatible with the PureMVC gestalt because the controller region doesn't don't talk to services, the model does.

The idea is that we want to present the data model to the rest of the application as if it were always present, and further, that it maintains its own integrity. The Proxy pattern achieves this and has the responsibility of hiding the fact that data may not in fact be present, and that requests for data that isn't there may involve remote service calls. Authentication is just another part of the model. Credentials are set, and if correct, the model should respect your requests for data, otherwise it may send notifications indicating that the credentials are not valid.

Of course we know that some use cases against the model are going to be async and some sync.

If the use case is async, we set a property or call a method on a Proxy, then the use-case is continued when the Proxy sends a notification, either predetermined by the proxy, or one we tell it to send when it's ready.

If it's synchronous, then we just set and get.

The nice thing about the async use case being continued by a notification is that multiple mediators often listen for the same notification, allowing them to do their own state management in response, rather than concentrating knowledge in a Command. This makes the view more reusable. It also leads to less state representation needing to be done on the model.

But never do we concern ourselves (in the controller and view regions) about services. This is fully a part of the model region in this implementation of MVC. Delegates may be used but by Proxies, not Commands. Thus, though the idea of an asynchronous command chain may make sense in another framework, it doesn't really fit here.

However, there is a quantifiable upside to this implementation, and that is that the model becomes portable. In the upcoming courseware there is a concrete example of this, where we share the same model package between two separate applications. Thus the model (and its communication with services) needs to be as much of a black box as possible. The job of Commands and Mediators then is merely to implement a unique view and carry out different use cases against a common model.

-=Cliff>

[graphex]

AsyncToken does work well for relatively simple cases - like communicating what to do after the model completes an operation. However, when you need to schedule several things to happen in order, it gets really dicey. Either your token takes on a life of its own (essentially becoming a command class itself), or you end up trying to follow notification responses through several iterations to figure out what the actual result of the original call will be.

Let's take an example of the startup of an application which needs to get information from multiple sources. My goal is to centralize the list of things that need to happen in order so that token chasing is kept to a minimum for anyone later analyzing the code.

Let's assume for the purposes of discussion that this is happening in Flash instead of Flex, and that we want to try to maintain the "pure actionscript" principle and avoid Flex-specific constructs.

Let's say my theoretical application has to:
1)Load data from flashVars into the model to get the URI of its datasource (synchronous)
2)use the loaded URI to download an XML data file (asynchronous)
3)use the information in the loaded data file to download a splash screen (asynchronous)
4)fade in the splash screen (asynchronous)
5)parse the rest of the loaded data and generate navigation from it, using assets referenced in the data file (concurrent asynchronous with #4)
6)when both 4 & 5 are complete, display landing page.

Using AsyncToken pattern instead of a Command pattern to perform the list above is possible, but trying to follow the chain of tokens and responders would get very confusing for anyone analyzing the code later on. Especially for something like this, where the model is acting somewhat like a view, it seems to me that a Command pattern is able to orchestrate asynchronous sets of tasks much more elegantly. I say the model is acting like a view in this case because the application needs to respond appropriately to server (or other asynchronous) actions much like it responds to user actions.

I'm not suggesting that the controller talk to services, but instead I am suggesting that the command classes of the controller be better able to orchistrate sets of actions which may branch and may include processing time. For example, what if the user is able to create a macro that the program needs to execute? or What if the application has a socket open to react to events from other users on a distributed network? or What if the user needs to fill out a wizard to perform some task, where each step of the wizard is dependent on the previous step?

I know that PureMVC isn't FrontController, but it seems like an async-enabled command class would give you the best of both worlds. I don't think it has to act like Cairngorm's SequenceCommand where a delegate (similar to a proxy) has to call "nextCommand()", but it seems like there is a minor improvement to MacroCommand which could handle some of these cases better. Maybe an addAsyncSubCommand( IAsyncCommand ) would work, making it so that an IAsyncCommand responded to a particular notification from a proxy.

Essentially, what I'm after is a way to serially execute commands even when some commands are asynchronous. It is a common task for me, and in my first PureMVC app, even the simple application loading process paraphrased above got unnecessarily difficult to follow with the existing command class. IMHO, commands are the best place to set up orchestrations like that, but when you can't have a command that survives throughout an asynchronous process, sequencing gets much more complex and you have to worry about things like race hazards much more.

[maddec]

I understand your point Graphex, I have asked myself this question several times. I understand your explanation Cliff and I will stick to the practice you are advising. It works perfectly welll when proxy send notifications back when asynch operations are complete. But it is right that sometimes it is a bit difficult to track the chain and also to give names to notifications I think that our natural practice to gather things to make them more understandable plays a role here.

My question is:
What about putting this multiple asynch Notifications management inside a special kind of mediator, which only purpose may be to listen to notifications and send new one? Does this causes the same conceptual problems as having a special kind of async macro command? Or is it a better practice?

By the way, I think that chaining asynch transitions (I mean visual transitions), should be managed in the view region of the framework into a transition manager or something like that. Commands should only start a transition chain/sequence and let the transitionsManager or specialTransition mediator handle everything as a blackbox. It may send a notification when the sequence of transitions is finished. But such notifications should be used precautionously to avoid having visual transitions to call commands on proxy, which sounds to me not very clean and logicall, isn't it?

I wonder what is your recommandations on this point Cliff.
What would be the best practice to manage transitions and their chaining? (especially if they do not depends on built in Flex transition (between states), but on third parties Tweening packages). Should we use a kind of delegate on mediators to link to transition manager?

Thank you for your interest.
Best regards.

maddec

[puremvc]

Hi folks,

I concede graphex's point about tracing down the tokens possibly being hard to follow later.

I think the real answer to this is to build this command chainer you're talking about, and make it available as an off-the-rack utility for PureMVC.

Right now there is a demo in the soon to be announced repository that does something similar, but not a command chainer:
http://browse.puremvc.org/wsvn/wsvn/Demo_AS3_Flex_AppSkeleton

Daniele Ugoletti's flex app skeleton demo uses a 'StartupMonitorProxy', discussion of which you'll find elsewhere on these forums. There is documentation of this demo and everything else on its way, but see if this seems useful to you in this capacity or if really a command chainer is what you want.

If so and you'd like to lead the project (that is anyone on this thread), propose a spec for it as a utility (it would need an accompanying demo as well) and have a look at this post in Contributor Central for info about how I'm doing trying to roll this out. I'll grant commit access and set up a project with its own bugtracker/wiki, etc. http://forums.puremvc.org/index.php?topic=142.0

-=Cliff>

[maddec]

By the way, I think that chaining asynch transitions (I mean visual transitions), should be managed in the view region of the framework into a transition manager or something like that. Commands should only start a transition chain/sequence and let the transitionsManager or specialTransition mediator handle everything as a blackbox. It may send a notification when the sequence of transitions is finished. But such notifications should be used precautionously to avoid having visual transitions to call commands on proxy, which sounds to me not very clean and logicall, isn't it?

I wonder what is your recommandations on this point Cliff.
What would be the best practice to manage transitions and their chaining? (especially if they do not depends on built in Flex transition (between states), but on third parties Tweening packages). Should we use a kind of delegate on mediators to link to transition manager?

About managing visual transitions, what is your piece of advice on this point?
Is there a recommended approach complient to pureMVC best practices?

Thank you in advance for your answer.

[puremvc]

I believe some thought should be put into the matter of states/effects/transitions being managed.

Currently I do this at the MXML level within the component definitions. I often want to reuse an effect in other places within the application. but I don't really like the idea of just hanging it on the Model like a piece of data. For one, that makes the Model serve the View and its supposed to be the other way around. The Model is the thing being represented and should be transportable between multiple Views (applications). Therefore it shouldn't have View related baggage, such as Proxies to manage references to view state information.

I could see the possibility of a utility package of view state managers:
org.puremvc.as3.utilities.flex.viewstate.TransitionManager
org.puremvc.as3.utilities.flex.viewstate.EffectManager
org.puremvc.as3.utilities.flex.viewstate.StateManager
etc...

But what would these managers do? What specific responsibilities would they have? What collaboration partners? Keep in mind we may want to use third party tweeners (but not at the cost of robust support for flex built in stuff). And since this is a utility add-on for Flex based PureMVC apps, it can eschew the core framework 'language only' tenant and tie itself to the Flex classes.

Don't go rushing off to hack these out, set down with some index cards first. If anyone can put together a reasonable scope for this project to manage view state within PureMVC, advance it, and we'll start a project in the repository and dub you the owner. (and yes the repository doors will be open to the public soon, we're working furiously to get things in order...)

-=Cliff>

[maddec]

I believe some thought should be put into the matter of states/effects/transitions being managed.

I agree with you I think it is necessary to find solutions to gather things related to Visual Transitions, and if what is proposed is as efficient as PureMVC and work well with it, it may be a great step forward.
 
I think the introduction of states and transitions is a very valuable one, but it brought out also questions and strategical decisions. I currently also use official state-transition within mxml components. But I now need to go a step further for Flex projects and I feel that relying on Adobe transitions, makes me loose a lot of flexibility I have when I use thirdparties tweening packages, like Fuse in Actionscript 2 or Tweener. By the way, Moses Gunesch who is behind the famous Fuse kit made public a new project named GO Asap (Actionscript Animation Platform) lately, which objective is to propose a generic Animation Platform to gather every Tweening packages and other animation engines like Physic Engines and Collada animations. I made a blog post on this subject if you are interested in:
http://analogdesign.ch/blog/index.php/archive/goasap-go-universal-tween-framework/

It can certainly help if you want to use your own tweens and custom reusable transitions and not rely on Adobe's one only.

I think there is an other point where solutions should be found: Flash projects which have not a State system, I think that we need such State System, for example for project which use Papervision3D/Away3d, because it can quickly become very hard to manage every visual transitions and their effects with a 3d approach.

To work on an independent State system as PureMVC propose its independent Events system aka Notifications, may be a solution?
- For Flex project you may rely on Flex States only (which scope is to remain at the periphery of the framework into mxml components changed by mediators) and/or use these "Framework Wide States".
- For Flash projects you may use it as an equivalent of Flex states?
The advantage is that you would have everything centralised and not depending on Flex, so if you want to re-use a transition or a state sequence into another project you could easily whether it is Flex or Flash. This is just suppositions, I am not very sure of putting it here, but I felt it may be worth something... 

The GO Asap project may help to propose something generic about animation/transitions/states management to "plug" to PureMVC. I have not yet dig enough into it, so I cannot give further clues at present time.

Thank you for your interest.
 

 

[Henk]

Has there been any progress in async command chaining?

I'm facing a similar situation where a macro command needs to execute a number of simple commands sequentially. Each simple command talks to the model which in turn has one or more calls to the backend (log on, get configuration, etc), which are asynchronous.

The philosophy that backend interaction is hidden in the model is nice and simple, however, that doesn't mean we can ignore the fact that some commands  are asynchronous. When there is a sequential dependency between commands, there is a need to wait until one async command is finished before the second one is started.

In short, the current implementation of the MacroCommand needs async support.

Here is one possible approach. This is just to help the discussion.

A. Add an IAsyncCommand interface and a SimpleAsyncCommand class.

   public interface IAsyncCommand
   {
      /**
       * Execute the <code>ICommand</code>'s logic to handle a given <code>INotification</code>.
       *
       * @param note an <code>INotification</code> to handle.
       * @param responder an <code>IResponder</code> to report the result of the command.
       */
      function execute( notification:INotification, responder:IResponder ) : void;
   }

The SimpleAsyncCommand class stores its responder (the MacroCommand), and calls into the model proxy. When the model proxy replies with result or fault method, SimpleAsyncCommand passes it up to its responder (the MacroCommand). The MacroCommand implements IResponder and uses the result method to flag the command as complete and continue on to the next command.

In addition, since SimpleAsyncCommand can talk to one or more asynchronous model proxies, the model proxies will need a responder to reply to as well. This can be implemented in SimpleAsyncCommand base class with the default behavior to pass the result or fault method back to the responder (the MacroCommand).

        public class SimpleAsyncCommand extends Notifier implements ICommand, INotifier, IResponder
        {
         ...
        }

IResponder is defined in Flex. Since you aim for cross-platform portability a similar interface can be provided with PureMVC.

The same pattern repeats itself in the model proxy and in the model business delegates. Unfortunately, the model proxy cannot use the same implementation. Only one instance of the proxy is being cached by Model. If it would store the responder, a second call risk overwrite the responder if the first one hasn't replied yet. A Flex solution is using the Async Token as described in this thread: http://forums.puremvc.org/index.php?topic=68.0

A possible more portable implementation is to use an 'async command queue' in which stores the responder, and one or more business delegate instances with the corresponding response handlers. The async command queue can be used by the MacroCommand in calling one or more SimpleaAsyncCommand, by SimpleAsyncCommand in calling one or more model proxies, and by a model proxy in calling one or more business delegates.

I'm looking forward to hearing your thoughts on this approach and whether you think its worth developing it further.

[puremvc]

First, a clarification; there is no problem with instantiating a proxy mutiple times (with different names), having it implement IResponder, and having multiple calls out at once. Start with CafeTownsend demo to see a delegate talking to a Proxy. Extrapolate from there.

And next, the guiding principle of 'as simple as possible, no simpler' has equal pressure at both ends of it.

The dilemma is that even with that maxim employed at every turn, there are people out there without much design pattern background or OOP client design experience who are struggling with how to use it. And all the documentation in the world doesn't help because they feel it should be simpler still!

The idea of Commands as responders within PureMVC was ruled out from an early on. Core to the design is that Commands should be stateless, and should go away after execution.

Making them responders does two things that do go against this:

 * Adds references to the Command somewhere that might or might not get removed: the result and fault callbacks.

 * It often forces the Command to have to do more logic on a fault than necessary.

For instance, some operations should be retryable on fault, some shouldn't. It's not enough to just pop up an alert when a fault comes back.

PROBLEM:
Your app is on front page of Digg today, your service is being pounded, and it's collapsed. The clients are all timing out and popping up an alert box saying the service timed out. You know you have to move your hosting, but isn't there something you could do right now?

SOLUTION:
The calls for data need to be retryable, and when the service spikes, you can bump the number of retries in the configuration that the client reads and it'll be patient and stand in line, eventually getting its data. (Perhaps entertaining the user with a nice animation of say, a monkey in the meantime.)

QUESTION:
So now your Command has to do what? Fire off the request again that was in its execute method. From its fault method. And keep the number of retries around to know when to quit. Starting to keep state in this Command. Not good. So what? Create another Command that calls this one back and send a notification for it? Who keeps track of the call depth, notification type property cast to int perhaps? eh.... no.

ANSWER:
The knowledge that a particular service call is retryable and the stationkeeping for doing that lives in the Proxy. The job of the Proxy is to hide this stuff from the rest of the application.

Services are for data, and the knowledge of them lives in the Model region of the app.

The data is the core business of the Proxy, creating or fetching that data when requested, formatting and transforming it if need be, or handling interaction with the service that the data comes from.

The Proxy may even internally hold the instance of the service component as well, if it is the only class in the system that needs access to that service component. Otherwise a Delegate and a DataServiceProxy that holds the service component as a data property is a pattern that requires only one extra actor, the Delegate. One Proxy in need of data from a remote service can retrieve the DataServiceProxy, request a given service (ala Cairngorm's ServiceLocator) make a call on it and set itself as the responder. QED. There's really no need for the delegate unless you're really feelin' it.

And in the same fashion that every other event-driven thing happens in the system, the Proxy sends a notification when the data comes in or a fault if the service fails and retries are exceeded or not allowed.

It is natural for those familiar with Cairngorm to feel that the Command/Delegate/Service collaboration pattern makes sense here, but it simply does not fit with the principles behind PureMVC.

I don't want to sound discouraging; I haven't provided clear enough documentation about RemoteProxy use, and that's coming, I promise. That need to clarify here once and for all may be sounding like inflexible 'not invented here' thinking, but I assure you it is not. It is an effort to keep the core vision intact.

If you absolutely want a framework that uses Commands and Delegates to talk to services, the Cairngorm is in general release and follows this organizational concept.

-=Cliff>

[Rhysyngsun]

@Henk

(insert general disclaimer due to lack of rest and the fact that this is off the top of my head):

If you have a Macro command that needs to handle asynchronous events, you could break the command apart at the aasynchronous points, and have everything past that point be another macro command (and so on and so forth). You can then register each part of that original macrocommand (now macrocommands/simplecommands themselves) to certain notifications. Then you could simply have initial command activate whatever asynchronous Flex activity you're going to do in the proxies, then that particular command goes away since it's done executing. Once the proxy detects the asynchronous operation is done, it sends a notification that begins the next series of commands and so on until you are done.

In a way, the commands would be used as pseudo-delegates, they simply get executed when their notifications arrive. :-)

[Henk]

@Cliff: thank you very much for the elaborate response and the explanation. I understand (and agree) that keeping commands stateless avoids a swat of problems and that the model should hide the backend integration. There is a lot of elegance in the simplicity of it and I'd love to keep it that way.
Unfortunately, the problem remains. I am probably making things too difficult so let me try to describe a simple use-case. Hopefully you can suggest a more PureMVC'esc approach for it.

@Rhysyngsun: thanks as well, that is something I can try, but how to solve the problem of conditional subcommands?

The use-case is: create a user account, with optionally an email mailbox. If the request to create the user account fails then no email mailbox should be created.
1: Add command 'createAccount', which has a parameter that indicates whether a mailbox should be created as well. In addition there are two model proxies, one for the user accounts, one for mailboxes.
2: When executed, the command calls the user account proxy and adds the user. The mailbox can't be created yet until we're sure the create user account request was accepted.
3. user-account-proxy sends a user-account-creation-successfull notification. So far so good.
4. The user-account-creation-successfull notification is bound to the create-mailbox command.
5. The create-mailbox command invokes the mailbox proxy to create a new mailbox.

The problem is that the state of whether the mailbox should be created, is lost along the way.

I appreciate the excellent feedback. 

[Rhysyngsun]

You could always store a setting on the user account as to whether that account has an associated email box. When the account creation is successful, you then run a command that checks to see if this option is enabled in the account, and then if it is it generates another notification to start the mail box creation command. This could also work in reverse, that when the user disables the email option, the system dispatches a notification to delete the associated email account.