Bus design and execution flow

[thomafred]

Hi there!

I was wondering if any thoughts have been formed around the design of the bus and the layer execution flow?

From reading the docs I get the impression that execution flows from the top layers and down to the lower layers, and that the busses might resemble a stack with each task/step of a layer being a frame on the stack?

[ScottzCodez]

Something like RabbitMQ was suggested as a good implementation of the messaging system / Info flow tool.
I don't think a Stack would be a good implementation, It's Last in First out. First in First out would be better. (a Queue)
RabbitMQ might be overkill, not sure. There may be a simpler, more lightweight, and elegant solution to start with, that can be easily switched to RabbitMQ later if needed. (I just have entry level knowledge of this topic of message queue systems)

[samgriek]

NATS is another queue system that's quite lightweight and should work. RabbitMQ is a decent choice though, given that you can set up the queues as durable. If for some reason a layer or RabbitMQ itself crashes, it should be able to pick up where it left off when restareted.

A stack is not the ideal way to go.

[thomafred]

Interesting!

Is execution flow linear in the sense that higher-level layers are handing instructions to lower-layers to be processed. The lower-layer decided whether to give instructions to an even lower layer or return a response to the higher layer. The higher layer would be idle while the lower-layer is processing.

From your queue I understand that agents instead are more asynchronous?

[daveshap]

I would recommend AMQP or syslog. As far as execution order, it does not necessarily have to be linear or sequential (though it can be). Rather, I tend to think as each layer as engines that are operating simultaneously and in parallel. Constantly consuming input from the buses and generating output on the buses.

Today, as LLMs are slow and expensive, it may make more sense to have it all fire in sequence. However, in the future, LLMs will be fast and cheap, and so you can just treat them as text consumers, probably generating and reading millions of words per second.

At that point, you want to ensure that you're not generating too much noise. Signal-to-noise ratio will ultimately be the most important thing, which we discussed briefly internally. Each message on the buses (whatever the protocol or server is) should have some metadata:

1. Timestamp
2. Source layer, microservice, container ID, etc
3. Priority level or severity (probably a scale of 1 to 10 like QOS codes) This will enable easy filtration
4. A UUID or GUID
5. A thread UUID or GUID (e.g. a way to correlate related events for "train of thought" or whatever)
6. Modality, etc

Obviously this will change over time, but yeah, metadata is key

[thehunmonkgroup]

Priority level or severity (probably a scale of 1 to 10 like QOS codes) This will enable easy filtration

Gonna push back a bit on this and offer a concern and another perspective.

In David Allen's Getting Things Done, he cautions against using a priority ranking system when dynamically managing tasks, for some of the following reasons:

1. Priorities can change rapidly in a dynamic environment.
2. Relying solely on priorities can lead to a reactive mindset.
3. It may create a false sense of accomplishment by ticking off 'high-priority' tasks.
4. GTD emphasizes tasks based on context (e.g., location or situation) rather than strict priority.
5. GTD promotes identifying the "next action" for forward movement instead of prioritization.

My inclination would be more of a First Things First approach, using urgency and importance as the deciding factors (along w/ the context factor mentioned above). GPT-4 is already aware of this with no additional prompting:

ME:

Let's say you have four tasks to accomplish. Task 'tango' is urgent, but not important. Task 'foxtrot' is not urgent, but important. Task 'whiskey' is not urgent and not important, Task 'lingo' is both urgent and important. Think about urgency and importance, and how those concepts relate to what order you would choose to do the tasks. Then, tell me what order you would do the tasks in, and why.

GPT-4:

Given the provided information, here's the order I would suggest accomplishing the tasks, based on the urgency-importance matrix:

1. Lingo (Urgent and Important): This task is both urgent and important, which means it requires immediate attention. Such tasks are typically crises, deadlines, or problems that need to be addressed right away to prevent negative consequences.

2. Foxtrot (Not Urgent but Important): While this task isn't urgent, its importance means that it has long-term implications or benefits. By addressing these tasks after the urgent and important ones, you ensure that you're also focusing on long-term goals and not just putting out fires.

3. Tango (Urgent but Not Important): Even though it's urgent, its lack of importance means it might not have significant long-term value. It might be something that appears to demand immediate attention but doesn't necessarily contribute to long-term goals or values. Still, since it's urgent, it's placed before tasks that are neither urgent nor important.

4. Whiskey (Not Urgent and Not Important): These tasks are typically the ones that can be postponed, delegated, or even eliminated. They don't contribute to short-term or long-term objectives and can be distractions if prioritized over more important tasks.

[DataBassGit]

I like this a lot. You could scale this by combining the two, so that urgency and importance each have a numeric value, and assign weights to the attributes.

So let's say a task's urgency has a value of 9 on a scale of 1 to 10. Urgency is weighted at 1.1, meanwhile importance has a value of 7, and a weight of 0.8. You end up with the weighted urgency being 9.9, and the weighted importance being 5.6. The two combined become the tasks priority, which is 15.5. Meanwhile another task has an urgency of 7, and an importance of 10. Those weighted values become urgency 7.7 and importance 8. Priority becomes 15.7. Thus the higher importance, but less urgent task wins. You could do this with any number of metrics, not just urgency and priority.

[hexadecimal-REBOOT]

here are my recommendations:

1. Messaging System: Considering the pros and cons, RabbitMQ could be a solid initial choice for a messaging system. However, if you’re concerned about it being overkill, we could start with NATS for its simplicity and lightweight nature. And always transition to RabbitMQ later if needed.

2. Execution Flow: The execution flow doesn’t necessarily have to be linear or sequential. As you said, agents could operate more asynchronously. This will allow for better scalability and can accommodate future changes in the speed and cost of LLMs (Language Models).

3. Metadata: Including extensive metadata in each message is essential for proper functioning and debugging. This would include details like timestamps, source IDs, and so forth. Metadata will improve the system’s flexibility and make it easier to filter and sort messages.

4. Task Prioritization: Instead of a rigid priority scale, a more dynamic approach like the urgency-importance matrix could be considered. This framework allows for a flexible handling of tasks and can adapt to rapidly changing priorities, which seems particularly suitable for a complex, evolving system. Like ACE