On massively parallel coderacks

Here’s a question: how to make FARG massively parallel? I’ve written about parallel temperature, and here I’d like to ask readers to consider parallel coderacks.

Like temperature, the coderack is another global, central, structure. While it only models what would happen in a massively parallel minds, it does constrain us from a more natural, really parallel, model. Though I’m not coding this right now, I think my sketched solution might even help the stale codelet problem Abhijit mentions:

We need the ability to remove stale codelets. When a Codelet is added to the Coderack, it may refer to some structure in the workspace. While the codelet is awaiting its turn to run, this workspace structure may be destroyed. At the very least, we need code to recognize stale codelets to prevent them from running.

Consider that most codelets fit into one of three kinds: (i) they can propose something to be created/destroyed, (ii) they can evaluate the quality of such change, and (iii) they can actually carry it out.

Now, whenever a codelet is about to change something up, why add it to the global, central, unique, coderack? I don’t see a good reason here, besides the “that’s what we’ve always done” one. If a codelet is about to change some structures in STM, why not have (i) a list (or a set, or a collection, etc.) of structures under question & (ii) create a list-subordinated coderack on the fly? Instead of throwing codelets into a central repository, they go directly to the places in which they were deemed necessary. There are multiple repositories for codelets.

Why do I like this idea? First, because it enables parallelism of the true variety. Each of these STM-structure-lists-bound coderacks can be running in their own thread. If some crazy codelet wants to change some set of structures, it needs to find the proper coderack, or to create it during a run. This means codelets cannot interfere with STM structures to which they haven’t been assigned access; & the STM-structure-list-coderack will assure that only one codelet goes at a time.

Moreover, it helps us to solve the stale codelets issue, by simply destroying the coderack when something needed inside the lists is gone. If a structure is destroyed, and a codelet was waiting to work on it, the codelet–in fact all the coderacks associated with the structure–can go back to cyberspace heaven.

How about programming this thing? It doesn’t even have to be too hard. It can be done gradually, as all the coderacks can be running in simulated parallel and fully tested before venturing into the dangerous waters in which the threads lead us.

(I don’t know when I’ll be able to try this idea out, but hopefully, one day soon.)

Does that make any sense?

Explore posts in the same categories: Architecture, Author: Alexandre Linhares


You can comment below, or link to this permanent URL from your own site.

9 Comments on “On massively parallel coderacks”

  1. amahabal Says:

    Hi Alex,
    I do not disagree, but tell me some more. Let’s say that the program is solving “1 1 2 1 2 3 1 2 3 4”, just to pick something at random. How many Coderacks are we talking about? 5? 500? What are the coderacks at any given stage? Is there one per group (say, one for the last three elements “2 3 4”)? When this group is extended to be seen correctly as “1 2 3 4”, do we replace or modify the Coderack? Is there one Coderack per bond as well(say, 1 2 3 –> 1 2 3 4)? Per category (say, ascending group)? These are not just implementation details, I think.

    If there are too many Coderacks, so many, in fact, that the expected population of each is less than 2 codelets, we may not have gained much. you’d need a happy medium, somewhat between 1 and too many.

    Again, I do not disagree, just trying to understand your proposal better.


  2. Michael Says:

    We are talking about a rack for each object in the Workspace, I think. That’s how I’d do it. (this is a careful phrasing of “that’s how I’m doing it.”) And then we’re simply no longer talking about a rack, of course.

    Here’s how I see it. The Coderack is simply a list of possible things to try on a more or less random basis. I think it’s far cleaner to come up with action pressures directly. For instance, if we have a scan group a b c, then there is naturally some pressure to try to extend it to a b c d. That pressure ceases to exist when the group is gone, and it’s higher if the group is salient.

    But salience is just a matter of interest, or attention, or whatever you want to call it. Actually, I think a cleaner way to organize the activity of the entire model is simply to choose a salient object at random, then ask it, “what pressures do you feel,” then choose a codelet at random based on that pressure. There is no rack, per se.

    In situations where there are multiple codelets to break a given process up into discrete pieces (e.g. scout-assess-build), this “momentum” can, and should. be represented as a state of the object itself. In a scout-assess-build model, the scout codelet proposes an object (say), by which we can mean, “creates a Workspace object with state ‘proposed’.” Now that object actually exists already, but is special in that it can’t be used as a component unit in other objects. (This bit’s hand-wavy.) But it can still have a high salience or urgency or attention, and when we ask it, “what do you want to do today,” it can reply, “get built.” See?

    In this model there is no actual Coderack, meaning that the massive parallelism can take care of itself.

    This model of codelets is my next task for Copycopycat. Wish me luck.

  3. amahabal Says:


    Seqsee already does what you mention in Paragraph 3 of your comment. I call this the action fringe of the object (where object is a number, group, bond, or category, but that can be extended). {See “get-actions” in lib/SThoughts/SObject.pm, for actions on groups and on numbers. The code needs clean up, but might be somewhat intellegible.} This is over and beyond the Coderack, which is still needed. Not all actions are related to individual objects. Some are related to none, others to more than one, and thus you need a Coderack for those things.

  4. Ab,

    These codelets with zero objetcs, what do they do? Aren’t they all for “system maintenance”? (that’s my case at least, so I’m wondering if my system is lacking in some important flexibility).

    There are chunks, objects which include others, and relations, things that obtain some objects and create new ones from the original. If a specific relation wants to change an object X inside a chunk C and an object Y outside of it, we will have a (C,Y)-coderack. Neither the chunk nor the 2nd object can be altered without entering this particular coderack (and its thread).

    You point out an interesting problem: perhaps 1000 coderacks with only two codelets might arise? Well, that is true.. it could be. But think about this: current processors are already at 45 nanometers… they’re getting to the limits, which makes me feel that soon, maybe 5 years, we’ll have 64 processors on a regular 1500 dollar noteboook.

    And if that isn’t enough, just google up “GPGPU” and “stream processing”.

  5. Abhijit Mahabal Says:

    The Zero object codelets could include codelets to estimate if we have a solution, check progress, and so forth. These are global tasks…


  6. amahabal Says:

    and also bond scouts and group scouts in Copycat, which are “global”, in the sense that they first choose some object to act on. They are not born attached to some object.

  7. Michael Says:

    In my model, the solution and progress would be units in the Workspace. Thus there is still a place to attach a Codelet.

    Likewise scouts — I regard it as an artifact of the code organization that bottom-up scouts don’t have objects attached when they’re posted. In my way of thinking, to determine an action, I always select a unit first, then determine what kinds of action it might want taken. I think this should work — it should be equivalent, really, just organized differently.

    But I’m happy to see your intuition is going in the same direction as mine, with your action fringe. That’s heartening.

  8. amahabal Says:

    I think your way might be cleaner. Seqsee has changed over time, and some of what it has is vestigial. If I redo, I might move more things to the action fringe.

  9. […] I’ve been thinking about massively parallel FARG, distributed temperature, and distributed coderacks: Now, whenever a codelet is about to change something up, why add it to the global, central, […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: