Brill Middleware is designed for developing transaction web sites and web sites that have dynamic content. Brill Middleware is the Web Sockets equivalent of REST but uses a “low code” approach and the server can push content to the clients.
Brill Middleware supports two way communication of JSON messages between client UI components and the server. The Middleware uses Web Sockets, not HTTP. A Message Broker runs on the Client and supports a “publish-subscribe” messaging model.
Components can subscribe to a topic and wait for someone else to publish content to the topic. On initial subscription or when a topic is updated, the component is notified of the content. Topics can be local to the client or span between the client and server. When content is published to a server topic, all the clients that have a subscription are notified and each client Message Broker will notify each component that has a subscription.
Brill Middleware differs from REST in a number of ways. With Brill Middleware there’s software running on both the client and the server. A component can make Subscribe and Publish requests to the Message Broker. A component provides a callback that is called when new content arrives. This is a much higher level interface compared to a HTTP client library and REST.
Other differences from REST are that the TCP/IP socket connection is held open for the duration of the session rather than opened and closed for each request. This reduces latency as a socket connection doesn’t need to be established for each message. The message content is pure JSON and there are no HTTP headers and no HTTP concepts like GET, PUT or POST. It’s just “publish” and “subscribe”.
The sending of messages is decoupled from the receiving of messages. A subscribe request could result in a publish half a second later and five minutes later another publish with updated content. To support dynamic updating using REST requires some form of polling, which could potentially drain the battery on a mobile device. With Brill Middleware, there’s no polling but the server can still push new content to the clients.
Brill Middleware supports passing of data between components as well as between components and the server. Brill Middleware can be used instead of libraries like Redux. A “publish-subscribe” model is simpler than Redux and doesn’t require code to be written for Actions and Reducers. Brill Middleware provides a “low code” solution. An extra field can be added to a database table and the data made available to a UI component without requiring any code to be modified.
A component is a React component that uses the Message Broker to communicate with other components and the server. A component will often subscribe to one topic and publish to another.
In this example we are going develop a component that displays the live results of a football match. At the start of the match the results will be displayed as a text string. For example:
Manchester United 0 : Chelsea 0
When a goal is scored, the new results will be published and the display updated:
Manchester United 1 : Chelsea 0
This is the component for displaying the results using React and Typescript :
this.state.text
} }]]>The topic is passed into the component as a prop. When the component mounts, it subscribes to the topic and provides a callback method. The Message Broker sends a subscribe event message to the server. The server gets the content for the topic and sends it back to the Message Broker using a publish event message. The Message Broker calls the callback method and the text in State is updated and in turn React calls the render method to display the new content.
If someone publishes a new score to the topic, the callback method gets called a second time and the display gets updated with the new score.
This is how the component might be used in the render method of another component:
The topic specifies the URI on the server of JSON containing the score. Ideally we would store the topic name in a config file or the CMS, so that it’s not hard coded and can be changed easily.
How do we update the score? We could have someone watching the live match and provide a web page for them to enter the new score. The score could be updated with the following method:
This will result in the server updating the score text in /football_scores/matchScore.json and
publishing the new content to every client that has a subscription.
Another option is that the server could publish a new score as a result of receiving the data via an external feed. From the point of view of our Text component, it is totally unaware of who the publisher is.
The good thing with our Text component is that it's fairly generic. We could use is for displaying a share price, the latest news or the temperature say.
Messages are sent between the client Message Broker and the server using JSON.
This is an example of a message the client might send the server to subscribe to a topic:
Should the topic be available, the server might send the client a message of:
When the Latest Story is updated, the server will send all the subscribed clients a message with the new content. When the client no longer needs to display the latest news story it can unsubscribe:
The messages are simple and can be sent from the client to the server or server to client. There are only four fields value that are specified in a message. These are event, topic, content and filter.
The Event field specifies one of the actions of “subscribe”, “publish”, “unsubscribe” or “error”.
subscribe - Sent by the client to the server to subscribe to a topic.
publish - The client can publish content to the server or the server can publish content to the client.
unsubscribe - Sent by the client to the server to unsubscribe from a topic.
error - Sent by the server or client to notify the other party of an error.
The Topic field is effectively the resource or subject that the message is about. The topic is a URI. The following are examples of topics:
json:/my_app/Pages/home.json
file:/my_app/Resources/title.txt
query:/my_app/Database/employeeQuery.sql
Topics can also be local to the client. For example:
app:/form.first_name
app:/form.dob
The Content field holds the topic content as JSON and is used when publishing. Content can be an Object, String, Number, Boolean, Array or Base64 encoded data.
This example shows publishing the results of a database query:
This example shows publishing of binary data using Base64 encoding:
For a topic that contains a large amount of data, a filter can be used to specify a sub-set that the subscriber is interested in. A filter can also be used to supply SQL query string parameters. For example:
The server uses topic names that are fully qualified URIs. For example:
json:/football_app/football_scores.json
On the client the same file might be referred to using the partial URI of:
/football_scores.json
Before the topic name is sent to the server, the Message Broker on the client will append file:/ and
the application name. The convention is that the application name is always the first part of the path.
The sections below detail each of the URI scheme’s that are supported:
On receiving a file subscription the server will get the file and publish it to the client. When a client publishes a new version of a file, all the other clients with a subscription are notified. Files can exist just as normal file system files or they can be held in a Git repository. Git provides a source control system and audit history. Git can be used to share files across multiple servers. Files are returned as Base 64 encoded content.
Example: file:/brill_cms/Pages/home.json
A path or directory can be specified. This will result in the publishing of the path tree. This is useful for the
application to find out what topics exist. An object is published containing the tree. A subscribe request for
the topic file:/storybook/Pages/components/ would result in something like:
This is the same as file: but the content is returned as JSON. The file extension must be
.js or .jsonc for JSON with comments. Using json: is more convenient
for an application than using file: when the content is JSON.
Example: json:/translation_app/Resources/dictionary.json
Executes a SQL query on the database. A filter can be used to provide query parameters and limit the number of
rows returned. The content is returned as JSON. The file must have an extension of .sql
Query parameters can be specified in the SQL using a filter field name preceded by a colon. e.g.
:row_count This results in the insertion of the parameter using a Pre-Prepared Statement. This is
safe and immune from SQL injection attacks.
Pre-Prepared statements restrict the locations where query parameters can be placed. You can’t specify a column name using a query parameter. To get around this, place a double colon in front of the filter field name. This will result in substitution before the Pre-Prepared Statement is created. To prevent SQL injection attacks, the query parameter value is checked to make sure it doesn’t contain any spaces or special characters.
A single colon query parameter should be used wherever possible and the double colon only used for column names and “order by” values.
Example: query:/hr/Database/employeeQuery.json
employeeQuery.json
Executes JavaScript on the server in a sandbox. The result is published as JSON. The file extension must be
.js . A component can specify parameters to be passed into the JavaScript using a filter. Normally
a single subscribe request results in a single publish. However the component can change the filter, in which
case the JavaScript is re-run and the callback called again.
If the JavaScript is updated as a result of a publish, the JavaScript is re-run and the components callback called again. In some instances this might not be desirable, in which case the component needs to unsubscribe from the topic on the first call of the callback.
The JavaScript runs in a tightly restricted sandbox on the server. The sandbox can be configured to allow the JavaScript to access Java methods. A Java method can be provided to allow access to the database or to do just about anything that can be done on the server.
JavaScript is used mainly for handling complex database queries and as a means of accessing external systems via a Java method supplied to the sandbox.
Example: javascript:/trading_app/ExternalSystems/getTradeData.js
Performs an admin operation such as user authentication. The client will subscribe and specify the authentication credentials in the filter. The server will either send a single publish message to the client or an error message.
The server authentication code may need to be changed to meet the specific requirements of the application. Care should be taken to ensure that messages do not contain passwords in the clear.
The admin scheme can also be used for other application specific admin tasks.
Example: auth:/trading_app/authenticate
The server makes a HTTP/HTTPS connection to the URI/URL. When there’s a filter specified a HTTP POST is performed or a HTTP GET when there’s no filter. With a POST, the content is supplied as “application/json” data. Ideally the REST endpoint will return “application/json” content. Content other than “application/json” will be returned Base64 encoded.
The server replies to each client subscribe request with a single publish response. If the response status code is not HTTP_OK, an error event will be sent to the client.
Example: http:/localhost:7000/products/food
The topic is local to the client and no messages are sent to the server. Any topic that doesn’t start with
a “/” and doesn’t start with <scheme>:/ is treated as a local topic. For example
form.name is a local topic. As a convention, slash ( “/” ) characters are not used in
local topic names. Instead dot (“.”) characters are used. This makes local topics easily
distinguishable from other topics.
Local topics are used for components to pass data between themselves. For example, a user input field can publish the users input to a local topic. When the users clicks a submit button the component can subscribe to the data and pass it onto the server.
Example: local:/claim.form.amount
Other scheme such as mailto:, ftp:, fax: or bitcoin:. can be used. Obviously some server code is required to
support any new scheme. The developer can also use their own scheme names. For
examplestream:/library/training_film.mp4 could be used and the data published using the WebSockets
Binary mode.
The client Message Broker maintains a list of subscriptions and component callback methods and the server also maintains a list of subscriptions. When a topic is updated, interested clients are sent publish event messages.
It's very important that components unsubscribe from topics they are no longer interested. Otherwise unnecessary publish event messages are sent from the server to the client and the Message Broker will attempt to call callback methods that are no longer valid.
In some instances a component might not want the display to be updated as a result of a second publish of the topic. To stop further updates, the data loaded callback can unsubscribe from the topic.
With REST, best practice is for the server not to hold any session data. This means that requests can be directed to any node in a cluster. Not using session data does however make it more difficult to track authenticated users. A token such as a Java Web Token (JWT) needs to be issued on authentication and included with every request. The server has to trust the client not to modify the JWT or pass it on to someone else. But the server can’t trust the client, so then a list of valid JWTs has to be maintained and replicated across the nodes in the cluster. All very complicated.
With Web Sockets, there’s a single connection that remains open between the client and server for the duration of the session. The load can be shared across a cluster but each client only talks to one node. This means there’s no need to replicate session data amongst the nodes of a cluster. Both the client and server are aware immediately of when the Web Sockets connection is closed, unlike with HTTP and REST. The client Message Broker attempts to re-connect immediately after a connection is closed and would connect to another node in the cluster.
Given the way Web Sockets work, Brill Middleware uses session data to hold the username and privileges. This is simpler and more secure than using tokens and also works with a cluster.
Applications have different security requirements. Rather than impose a ridged security framework, it’s left to the application developer to implement the required security hooks and checks. The session data can be used to hold the username, users groups and permissions. Thought needs to be given to which users are allowed to subscribe to which topics and also which topics they are allowed to publish too.
Publishing of JSON, SQL and JavaScript files needs to be restricted. One option is to hold the files in git and to have the Production Server use the MASTER branch. Changes can be prohibited to the MASTER branch. Changes are made on the Development Server using the DEVELOPMENT branch. When the changes have been tested and approved, they can be moved to Production by merging the DEVELOPMENT branch into the MASTER branch.
A Production Server should always use https and have a valid certificate. A hacker can view the Web Sockets messages, as they could REST API messages, using browser developer tools. This is actually useful sometimes for debugging. Passwords and sensitive data should therefore not be passed in the clear. One option is to use Secure Remote Password (SRP), so that the password never leaves the client.
Brill Middleware includes a set of custom Spring Boot annotations to support writing Web Socket Event Controllers. These are similar to REST Controllers but instead of using @RestController they use the annotation @WebSocketController. Endpoints for messages are annotated with @Event.
This is an example of handling an authentication message:
Note that unlike with a REST controller, the method doesn’t return a response. It calls a method to send a message to the Client and can send multiple messages or indeed none at all.
This annotates the class as a Web Socket Controller. This annotation is essential and without it none of the event methods will get called.
This annotates a method specifying the event and topics for which the method will be called. In the example above
the event is "subscribe" and the topic has to match the regular expression
"admin:/.*/authenticate" for the method to be called.
The underlying management code checks that a single event method is called for each message received. An error is logged if either no event handler matches or more than one event handler matches.
This is a parameter annotation that provides access to the WebSocket session for sending messages and saving data to or reading data from the session data.
This is a parameter annotation that provides access to the JSON object containing the message.
Brill Middleware takes a “low code” approach and seeks to minimise the amount of code the application developer has to write.
With REST, typically a separate endpoint is developed for each database operation. Hibernate or some other Object Relational Mapping library is used to persist Java objects. A fair amount code is required. When a new field is added to a database table, changes have to be made in a number of the places and a new release rolled out.
With Brill Middleware, everything is handled as JSON. There’s no conversion of database results to Java Objects. No persisting Java Objects to the database. All objects are held as JSON. This means we can write a database query that gets results and these are returned to the client as JSON with no Java Objects involved. We can add a new table to the database, add a new SQL query file and make the new table available to the client application without any code changes, releases or restart of the servers.
On the client side, the Message Broker provides an easy to use interface for UI components to subscribe, publish and unsubscribe. Objects are provided as JavaScript objects.
The reference implementation was developed using React and Typescript. The server is a Java Spring Boot application running on Tomcat. The specification of the Middleware is generic and the Message Broker can be used with React, Angular or any other JavaScript framework. The server could be implemented using NodeJS or any server that supports WebSockets.
The server functionality can be tested independently of the client application using a Chrome extension called WebSocket King.
There are other Web Sockets test tools available such as Postwoman, the WebSocket equivalent of Postman. The advantage of WebSocket King is that it's simple and easy to use.
All modern web browsers support WebSockets. The first support was first added in 2011, so WebSockets have been around for a long time and are widely supported. There's a fallback mode that uses HTTP and long polling just in case. This is used when the web browser doesn’t support WebSockets or where a firewall or network blocks WebSockets.
Key features provided by Brill Middleware are:
Publish subscribe model
Pushing of data by the server
High performance and low latency
JSON throughout
Topic specified as a URI
Supports communication between UI components and the server
“Low code” approach