- MaXX Interactive Desktop Architecture
- Layered Architecture Details
- MaXX Settings
- MaXX Links
- MaXX GPES
- MaXX Scope
- MaXX Monitor
- MidView Overview - Local and Distributed Deployment
- MidView Internal - Component View
- MidView Internal - Component Interactions
- MidView Internal - Containers
- IconCatalog and FM Renderer
- Icon Design
- Best Practices
MaXX Interactive Desktop Architecture
Goals and Requirements
- Provide a secure modern and robust computational environment for smart applications and services.
- Support both local and distributed deployment.
- Be modular, expandable and flexible while supporting a loosely couple design.
- Do more with less. Meaning be smart in how resources are used and keep it in mind.
- Be as fast and efficient as possible. It is a balancing act and it always depends on the use-case.
- Support CPU architecture specific optimizations.
- Support Hardware Acceleration (GPU decode/encode & rendering).
- Support high performance asynchronous messaging for inter-application communications.
- Can run on different OS and CPU architectures (Intel/AMD, ARM and RISC-V).
The MaXX Desktop's global architecture is based on the following three (3) principles in order to ensure the above goals and requirements are met.
- Support a multi-layers style architecture for applications and services separation with clear boundaries and responsibilities.
- Support a messaging base architecture for both inter-process communication and asynchronous event-driven execution.
- Support Shared Memory to prevent unnecessary copy/duplication of information when doing local (same machine) inter-process computations or when handling/displaying images over a local X11 session.
The architecture is inspired from multi-layered Enterprise class system where each layer defines precise responsibility and its application/service (a.k.a. component) exposes functionalities or behaviours. At the heart of the architecture are clear communications channels between components and their respective layers and well defined data-contracts exposed by each of them.
Overall, this design strategy will improve robustness of the Desktop experience, provide solid foundations to build upon, allow a clean separation of concern, modularity and overall re-usability. Another way to see MaXX's architecture design is to look at micro-services architecture, but from a Desktop application perspective where each layer of the architecture is composed of specialized micro-services performing specific tasks.
Layered Architecture by responsibility
The MaXX Desktop architecture is divided into three (3) responsibility layers from which MaXX's application and service can be built. Below are the layers in question with a short description for each:
- User Experience/Presentation - layer performing visual-oriented tasks like displaying User Interfaces and/or capturing user's input. Components and/or applications on that layers are communicating with the Desktop Support layer for computation and resources access via proxy like services.
- Desktop Support - layer provides desktop computation support while providing an abstraction-layer to various Back-end Services, where most of the actions are taking place. This layer exposed some of those computations as functionally aggregator (composition and proxy design pattern) where a specific Desktop Support functionally is realized by utilizing one or more Back-end Service and an orchestration service that can coordinate the execution of other services/components.
- Back-end Services - layer is where most of the actual work is performed by low-level services/components. This layer can only communicate with components/services from the Desktop Support layer. Among the functionalities exposed by this layer are: hardware and application monitoring from MaXX Monitor, file-system accesses and configuration management via MaXX Settings.
The diagram below illustrates the current MaXX Desktop architecture
MaXX Links (Inter-Layer Communication)
The MaXX Desktop architecture is planning tree(3) means of inter-layer communication mechanisms ensuring security and maintaining separation of responsibility.
CLI or Command Line Interface - allows a command-line driven client/server like interaction where the Client CLI initiate a CLI session with the Server CLI counterpart and sends one or many CLI commands. The communication is obviously synchronous and based on a request/response semantic that is redirected through the POSIX standard I/Os of the CLI Client application. Nothing prevents the Server CLI to emit an event during the processing of a CLI command. The CLI mechanism is great to provide a fire-and-forget interaction semantic.
HTTP or REST Interface - allows a Web/REST driven client/server like interaction where the HTTP Client sends a request to a HTTP Service. The communication is again synchronous and based on a request/response semantic using HTTP/S and predefined data types. Nothing prevents the HTTP Service to emit an event during the processing of the request. This mechanism is great to provide a a full features API style integration.
Messaging - allows synchronous, asynchronous or event-driven interactions where a producer create a message (as event or notification) and send it to a communication channel. One or many consumers are at the receiving end of that channels allowing either point-to-point or publish-subscribe delivery mechanism. The payload of a message can either contain real data (passing by value) or provide a secured shared-memory location where the data can be retrieved (passing by reference).
The diagram below illustrates the inter-layer communication mechanisms.
The communication between components and layers traversal will be provided by MaXX Links, which supports modern multi-protocols synchronous/asynchronous messaging patterns. Things such as: load balancing, tasks distribution, high availability, decoupling client-server with simple consumer/producer semantic are all possible (and in many cases rather simple to implement) through Messaging.
Messaging also introduce the ability to support a polyglot code base where components integration are performed via high performance messaging channels while supporting both local and distributed environment. It does not really matter anymore in which programming language the component/service is written, as long as it supports Messaging ZeroMQ and complies to predefined data-contracts.
Through MaXX Links, any C/C++/Python or Java application can be integrated into the MaXX Desktop environment or new modern features can be added to an existing application very easily.
The Messaging approach as been proven to yield excellent values by allowing ongoing improvement of components, total decoupling between producers and consumers and bring robust asynchronous and event-driven capability. As long as data-contracts are respected, changes documented+communicated, with versioning and backward compatibility supported, this is one of the best way to do inter-application communication.
The MaXX Interactive team bring more than 2 decades of real world expertise in High Performance Messaging Systems. That must count for something :)
We utilize this 'passing by reference' strategy quite heavily in GPES (General Purpose Execution Service) as a very efficient way to consume computation results without the net-impact. In some cases, all the work/computation, memory allocation and display are all performed by the GPU.
Work in Progress...
Layered Architecture Details
Layered Architecture by responsibility
As we previously saw, the MaXX Desktop layered architecture is divided into three (3) responsibility layers from which a MaXX aware application or service can be built.
Here are the layers in question with a list of components (applications/services).
User Experience & Presentation
This layer is performing visual-oriented tasks like displaying User Interfaces and/or capturing user's input. Components and/or applications on that layers are communicating with the Desktop Support layer for computation and resources access via proxy like services.
User Experience & Presentation Components
|5Dwm||Enhanced Motif Window Manager.|
|Toolchest||Desktop Application Menus and Launcher.|
|IconCatalog||Visual and Interactive Application catalog.|
|File Manager - fm||Visual and Interactive File Manager.|
This is layer provides desktop computation support while providing an abstraction-layer to various Back-end Services, where most of the actions are taking place. This layer exposed some of those computations as functionally aggregator (composition and proxy design pattern) where a specific Desktop Support functionally is realized by utilizing one or more Back-end Service and an orchestration service that can coordinate the execution of other services/components.
Desktop Support Components
|MaXX Launcher||Smart Application and Service Launcher.|
|MaXX Scope||Smart Application and Service Orchestration for multi-core system with CPU cores partitioning.|
|MaXX Session||User Desktop Session Manager.|
|MaXX GPES||General Purpose Execution Service - Centralize and Unified Task Execution Environment.|
This layer is where most of the actual work is performed by low-level services/components. This layer can only communicate with components/services from the Desktop Support layer. Among the functionalities exposed by this layer are: hardware and application monitoring from MaXX Monitor, file-system accesses and configuration management via MaXX Settings.
Back-end Services Components
|MaXX Settings||System Settings and User Preferences Management Service.|
|MaXX Monitor||Centralized Hardware and Application monitoring with metric aggregation.|
|MaXX File Service||High Performance and Multi-threaded File System Service.|
Work in progress...
MaXX Settings is a dynamic configuration management subsystem designed from the ground up with simplicity in mind while not sacrificing flexibility and extensibility. MaXX Settings comes with its own CLI interface allowing simple management, automation via scripting, inline-query and easy application integration. MaXX Settings also provides Java and C++ binding making it super easy to integrate within most modern applications. MaXX Settings allow the definition of System wide setting, we call them Instruments, and user’s overridables called User Preferences.
Here are the requirements that MaXX Settings must strive to enforce or provide:
- Retrieve information as fast as possible (flat lookup speed curve).
- Provide different levels of verbosity (admin vs normal user).
- Software design based on current/modern technologies while future proofing the code with a component/modular approach.
- Use SOLID Principles (most): Single responsibility, open-close, interface segregation and dependency inversion.
- Favour simplicity over complexity.
- Support multiple OS.
- Provide a Command Line Interface (CLI) to administer, query and set data.
- Be human friendly with its interfaces.
- Provide an API for C++ and Java clients.
- Provide user based authentication.
- Support UTF-16 for its internal String encoding.
- Support hierarchical data structure suited for a dynamic typed configuration management system for Desktop, Application and FileType instrumentation.
For the complete Technical Specifications, refer to this [online] document
The Architecture design on which MaXX Settings is built follows the Client/Server model, where the Client is represented by Users using a CLI type interface or Applications using an API both interacting with the Server. The Server provides the functionality to manage instrumentation on behalf of the Client.
Development and Execution Platform
Java was selected for the first implementation of the Server for its richness in features, robustness, maturity and special affinity with server/service APIs and prebuilt components. The GraalVM was selected for its ability to compile Java byte-code into native code and run applications much faster. The optimizations offered by GraalVM have the added benefits to reduce startup and execution speed quite considerably.
Instrumentation Data Persistence
All information managed by the Server is persisted into regular files. No external dependency required for the database.
The database is implemented using fixed length field strategy to ensure simplicity but high performance. Saved information into the database is kept to a minimum as it is mostly to provide lookup mechanism.
MaXX Settings can be integrated using the CLI command line or via its C/C++/Java APIs.
Refer to MaXX Settings Framework documentation for more detail.
At its core MaXX Desktop relies on a robust high performance inter-process (not limited to process) infrastructure. Messaging is a natural choice for fast and efficient inter-processes communications, but yet simple. Enters MaXX Links, a multi-languages, multi-protocols, broker free, standalone, high performance messaging and inter-process communication library.
Here are the requirements for MaXX Links:
- support synchronous and asynchronous connections
- support multiple protocols
- support authentication
- support transport layer encryption
- support transport optimization for local messaging such as passing by reference using Shared Memory
- support message-driven architecture
- promote loosely coupling
- support connection patterns : one to one, publish /subscribe, router and dealer
- support distributed deployments
- provide a simplified implementation of the Enterprise Integration Patterns
- support for: C, C++, Python and Java
- provide an abstraction layer that could support different back-end providers
- support multi threading and signalling mechanism between threads
- must be light, low in dependency and broker free
Build or buy
The eternal dilemma... With the current state of affair, very small pool of developers, limited free time and so much things to do, it makes no sense to decide to build from scratch an entire messaging sub-system that must comply to the above requirements. On the other end, the build option could make sense in a possible future. For that reason, MaXX Links must be built with the future in mind and support some kind of abstraction layer allowing different back-end to be plugged in.
The decision was made to use ZeroMQ communication library as MaXX Links back-end provider and built an abstraction layer (wrapper) to ensure its ability to be back-end provider agnostic. ZeroMQ is a mature, robust and extremely fast messaging library to crossed on the boxes on our shopping list, and more. Applications build in C, C++, Python or Java (to name a few) can be integrated into the MaXX Desktop environment with ease.
Now that the messaging back-end has been taken care of, we can focus on the real
Since MaXX Desktop is built as a multi layered architecture of inter-connected components (applications and services), messaging plays a critical role at different levels and helps bringing everything together into an homogeneous, yet simple loosely couple Eco-system. Advanced features such as: load balancing, tasks distribution, high availability, thread signalling, decoupling client-server with simple consumer/producer semantic are all possible (and in many cases rather simple to implement) through Messaging.
A Message is a generic wrapper to some metadata and a payload that is transported between Endpoints over a Channel. A Message contains headers and a payload. The payload can be anything technically (more on that later) and the headers holds important information like: unique identifier, destination, creation timestamp, expiration, payload type descriptor and optionally a reply-to Channel. Headers are containing context information regarding the message's intent and its content.
The payload can be of any types, but for performance and efficiency sake, MaXX Links limits the payload type to Text, Bytes and SharedMemoryLocator. The Text type can be any sub-type of text such as XML, JSON or plain text. Bytes are used to transport raw information like an image or file, whereas SharedMemoryLocator allow to pass-by-reference larger chunk of data, within the same machine and without the Network-tax.
A Message Channel represents the "pipe" of pipes-and-filters high-level concepts of our Message Driven Architecture. Producers are sending Messages to Channels, and Consumers are receiving Messages from Channels. The Message Channel, therefore, decouples the endpoints from each other and provide an abstraction from the actual transport of the messaging sub-system. Channels also provide a convenient data collection points for intercepting and monitoring of Messages and the overall performance of the messaging layer.
Message Channel are either implementing Point-to-Point(queues), Publish/Subscribe(topics), Router or Dealer semantics. With Point-to-Point Message Channel, there will be at least one producer and a minimum of one consumer. There is no limit on the number of producer vs. consumer, but rather by the technical requirements and common sense. The most simplistic use-case is one producer and one consumer connected via a channel. This model can be pushed to one producer and many consumers connected via a channel that serves as a kind of load-balancer for parallel execution or router based on some Header based rule. Publish/Subscribe channels, on the other hand, will attempt to broadcast each Message to all of its subscribers, such as in a an event or notification delivery feature.
A Message Endpoint represents the "filter" of a pipes-and-filters architecture. The Endpoint’s primary role is to connect your business logic code with the messaging framework in a non-invasive manner. In other words, the business logic code shouldn't be aware of the Message objects and Message Channels. Similarly to the a HTTP Controller class in the MVC paradigm, where the Controller handles a HTTP request, extracts incoming data, transforms it and forwards it to the business logic for execution, then does the reverse with the result of the execution, build a responses and sends it back. An Endpoint is doing pretty much the same things but under a standardized messaging framework.
As with the Model-View-Controller paradigm for web applications, MaXX Links goal with the Message, Channel and Endpoint approach is to provide a thin standardize dedicated transport layer that extract and translate data from an inbound Message received from an incoming Channel, then invocation the business logic attached to the Endpoint and translates back the response into an outbound Message, that is sent to the outgoing Channel. That was a month full, but in essence pretty much it! In a successful Message Driven Architecture implementation, the value is created by this separation of concern and the measure of its success is by maintaining the business logic totally clueless of its surroundings.
Endpoints are asynchronous in nature and are ran in threads. Asynchronous multi-threading provides a much smoother and less spiky workload across all allocated CPUs/Cores. MaXX Monitor and MaXX Scope will ensure that is the case.
One of MaXX Links main goal is to simplify the development of inter-connected components and applications through a set of already made components that accelerate the development cycle. Ultimately this means no one should ever have to re-implement low-level communication, transport layer, consumers nor the producers, but rather instantiate or extend existing components. Then plugin the business logic to Endpoints and go.
MaXX General Purpose Execution Service, or GPES for short, is a high performance tasks execution service that takes its inspiration from High Performance Computing (HPC) system. MaXX GPES provides robust and consistent computation solutions to common and repetitive tasks while providing high throughput and balanced utilization of system resources.
Work in progress... Feel free to share with us an idea or feedback. Come back soon :)
Here are the requirements for MaXX GPES:
- provide robust and secured execution environment
- centralize the execution of common desktop tasks with low latency (off-screen rendering, computation, etc)
- support POSIX multi threading library
- support Shared Memory for efficient exchange of large data set
- leverage GPU hardware acceleration for off screen rendering
- leverage CPU specific architecture optimization
- support MaXX Links for client connection (synchronous and asynchronous) and signalling mechanism between threads
- capture execution metrics and export them via API call
- portable to FreeBSD and alike.