References
This document explains all the API that zeeSQL provide to the users.
zeeSQL
is a Redis module, the command illustrate below are added on top of all the existing Redis commands. Using zeeSQL
means using Redis at the same time. You can use any Redis client, in any programming language, even the redis-cli
, to send the zeeSQL
commands.
This document refers to the latest API, but zeeSQL
commits to backward-compatible API.
For each command, it exposes first the name and then the syntax, and finally a brief explanation of what is going on inside the code.
Where is possible, it provides also an estimate of the complexity but since we are talking about databases not all queries have the same time and spatial complexity.
Finally, if it is appropriate the document also provides several references to external material that the interested reader can use to understand better the dynamics of every command.
ZEESQL.CREATE_DB
This command creates a new DB and associates it with the key.
The path argument is optional and, if provided is the file that SQLite will use. It can be an existing SQLite file or it can be a not existing file.
If the file exists and if it is a regular SQLite file that database will be used. If the file does not exist a new file will be created.
If the path is not provided it will open an in-memory database. Not providing a path is equivalent to provide the special string :memory:
as the path argument.
After opening the database it inserts metadata into it and then starts a thread loop.
Complexity: O(1), it means constant, it does not necessarily mean fast. However, is fast enough for any use case facing human users (eg create a new database for every user logging into a website.)
Examples:
This command created an in-memory database. Persistency is managed by Redis with AOF or RDB following the setting of your Redis instance.
This command created a database that uses a file as storage support, in this case /tmp/foo.sqlite
.
If the file does not exist, it is created.
If the file is already an SQLite database, it gets used immediately with all the data already loaded.
If the file is not an SQLite database, an error is raised.
See also:
DEL
This command is a generic command from Redis.
It eliminates keys from Redis itself, as well if the key is a RediSQL database create with ZEESQL.CREATE_DB
it will eliminate the SQLite database, stop the thread loop and clean up everything left.
If the database is backed by a file the file will be closed, but it won't be deleted.
Complexity: DEL is O(N) on the number of keys, if you are only eliminating the key associated with the SQLite database will be constant, O(1).
Examples:
See also:
ZEESQL.EXEC
The EXEC command is the main command of zeeSQL. It allows interaction with the database stored in the db_key
.
It takes as input the database db_key
, either a COMMAND
or a STATEMENT
, an optional series of flags, and an optional variadic number of arguments.
You need to supply EITHER a command (using the COMMAND
flag) or a statement (using the STATEMENT
flag), but you need one of them.
Command vs Statement
The command is a valid SQL string. The command SQL string can contain arguments in the form ?n
. Those arguments will be matched to the one provided at the end of the command. The first argument is bound against ?1
, NOT against ?0
. Arguments that are not provided will be bound to NULL
.
The statement can be created with the ZEESQL.STATEMENT
command. The arguments follow the same logic of the COMMAND variants.
Examples:
In this example, we create a new table.
Then we create a new STATEMENT to insert values, and we execute the statement twice.
We queried the values just inserted in the table executing another command.
NOW flag
By default zeeSQL
offload the SQL computation to a secondary thread. This free the main Redis thread and keep the Redis instance reactive.
The NOW
flags force zeeSQL
to run the SQL computation in the main Redis thread.
READ_ONLY flag
The READ_ONLY
flags communicate to zeeSQL
that the execution will not modify the database.
If the execution might modify the database, and the READ_ONLY
flag is passed, zeeSQL
will return an error.
Passing the READ_ONLY
flags allow zeeSQL
to not replicate the command. Possibly saving computing resources of the replicas.
Examples:
Here we correctly used the READ_ONLY
flag to communicate with zeeSQL
that the command will not modify the database.
zeeSQL
correctly executes the query.
In this other case, we ask zeeSQL
to modify the database while using the READ_ONLY
flag.
zeeSQL
correctly refuses to modify the database and returns an error.
NO_HEADER flag
By default zeeSQL
returns information about the result set. It returns the name of the columns and their type.
This information might not be useful nor desirable.
With the NO_HEADER
flag, only the result itself is returned.
Examples:
This is the default result from zeeSQL
. It reports the name of the columns (in this case number_as_integer
and number_as_string
and their type INT
and TEXT
).
If we are not intereted in such information:
the NO_HEADER
flags will omit it for us.
JSON flag
By default zeeSQL
returns its result as an array of array. This makes parsing the result a little complex in some programming languages.
The JSON flags instruct zeeSQL
to return a single JSON string as result.
Examples:
The JSON is valid JSON even if compressed:
Of course, this can be combined with the NO_HEADER
flag.
INTO stream
zeeSQL
can push the result of a computation in a Redis Stream.
This is desirable if you want to:
consume the result at a later time,
or cache the result,
or if the result is rather big and you don't want to send all of it over the network.
The INTO stream
option will inform zeeSQL
to push the result of the computation into the Redis STREAM called stream
.
The INTO stream
option is available only if the query is marked as READ_ONLY
.
The command executes XADD
to the stream.
If the stream does not exist a new one is created.
If the stream already exists the rows are simply appended.
The command itself is eager, hence it computes the whole result, append it into the stream, and then it returns. Once the command returns, the whole result set is already in the Redis stream.
The return value of the command depends on the result of the query:
If the result of the query is empty, it simply returns ["DONE", 0]
.
If at least one row is returned by the query the command returns:
1.the name of the stream where it appended the resulting rows, which is always the one passed as input 2. the first ID added to the stream 3. the last ID added to the stream 4. and the total number of entries added to the stream.
Using a standard Redis Stream all the standard consideration applies.
The stream is not deleted by zeeSQL, hence it can be used for caching, on the other hand too many streams will use memory.
The stream uses a standard Redis key, in a cluster environment you should be sure that the database that is executing the query and the stream that will accommodate the results are on the same cluster node.
This can be accomplished easily by forcing the stream name to hash to the same cluster node of the database, it is sufficient to use a stream_name
composed as such {db_key}:what:ever:here
. Redis will hash only the part between the {
and }
to compute the cluster node. 3. The result can be consumed using the standard Redis streams commands, two good starting points are XREAD
and XRANGE
.
The key of the stream elements are the tuple (type, column name)
separated by a colon :
.
Examples:
At first, we tried to push the result of a query that is not marked as READ_ONLY
and this correctly fails.
Above we execute a query that returns an empty result.
zeeSQL
simply communicates to us that the result is empty.
Then we push into the Redis stream foo_stream
the result of the query select * from foo
.
We do it twice, once using the standard return and the second time using the JSON return.
We pushed twice, a query that returned two results, so we expect 4 elements in the stream.
We can read the elements from the stream using the standard Redis stream interface. In this case, we are going to read only the first two elements.
Complexity: Besides the complexity of the query, the INTO stream
options add the complexity of adding each row to the Redis stream, which is O(n)
where n
is the amount of row returned by the query.
See also:
ARGS arguments
The ARGS arguments are used to pass arguments to the statement or command.
They are variadic, and you can pass as many as you need.
In the SQL, the first argument will be bound to ?1
, the second to ?2
, and so on. Please note that the first argument is NOT bound to ?0
.
If an argument is not bound, for instance, you pass a query with ?4
but only provide 3 arguments, then, that argument is bound to NULL
.
Redis works using a text protocol, all the arguments are encoded as text, hence the module is forced to use the procedure sqlite3_bind_text
, however, SQLite is smart enough to recognize numbers and treat them correctly. Numbers will be treated as numbers and text will be treated as text.
See also:
ZEESQL.QUERY
This command behaves similarly to ZEESQL.EXEC
but it imposes an additional constraint on the statement it executes.
It only executes the statement if it is a read-only operation, otherwise, it returns an error.
A read-only operation is defined by the result of calling sqlite3_stmt_readonly
on the compiled statement.
The statement is executed if and only if sqlite3_stmt_readonly
returns true.
This command is exactly like ZEESQL.EXEC ... READ_ONLY
however it can be executed against Redis replicas.
Complexity: Similar to ZEESQL.EXEC
, however, if a statement is not read-only it is aborted immediately and it does return an appropriate error.
See also:
ZEESQL.STATEMENT
This command manages zeeSQL
statements.
A statement is a pre-compiled SQL query, if you are going to execute your query over and over again, it is a good idea to make it into a statement. Under the hood it is a sqlite statement.
Statements can be used in the ZEESQL.EXEC db_key STATEMENT stmt
command and in the ZEESQL.QUERY db_key STATEMENT stmt
command.
Five different actions can be invoked with the STATEMENT command.
Create a new statement with the
NEW
option.Delete a statement with the
DELETE
option.Update a statement with the
UPDATE
option.Show the SQL behind a statement with the
SHOW
option.List all the statements with the
LIST
option.
The STATEMENT
command includes the NOW
flag. The NOW
flag forces zeeSQL
to execute the action in the main Redis thread. In standard operations mode, it should not be used.
NEW
The NEW
option takes as input the name to associate with the statement and an SQL query to compile.
The command compiles the SQL query into a pre-compiled statement, and associate it with the name.
The CAN_UPDATE
flag to the NEW
command, instruct zeeSQL
to behave as an UPDATE
if the statement name is already allocated to an old statement. Otherwise, without the CAN_UPDATE
flag, if the statement name is already used by a different statement, the command fails with an error.
DELETE
The DELETE
option deletes a statement.
UPDATE
The UPDATE
option updates a statement, associating the old name with the statement compiled from the SQL query.
If the name does not exists, UPDATE
fails, unless the CAN_CREATE
flag is provided. In such a case UPDATE
behave like NEW
.
SHOW
The SHOW
option returns the SQL query behind one statement.
LIST
The LIST
option returns all the statements and their SQL queries.
Both SHOW
and LIST
will report:
The name of the statement
The SQL query associate with the statement
The number of parameters the statement expects
If the statement is read only or not
Complexity: Operation on the statements happens in constant time O(1). Listing the statements happens in O(n) with n
number of statements present in the database.
Examples:
At first we create a new statement,
We can then list, the statement:
The statement can be updated, but we need to use the UPDATE
command or the CAN_UPDATE
flag.
We change select_1
to return a string and not an integer.
We create another statement, this time we create the statement with the UPDATE
command and the CAN_CREATE
flag.
See also:
ZEESQL.COPY
The command copies the source database into the destination database.
The content of the destination databases is completely ignored and lost.
It is not important if the databases are stored in memory or backed by disk, the COPY
command will work nevertheless.
This command is useful to:
Create backups of databases
Load data from slow, disk-based, databases into a fast in-memory one
To persist data from an in-memory database into a disk-based database
Initialize a database with a predefined status
Usually, the destination database is an empty database just created, while the source one is a database where we have been working for a while.
This command use the backup API of sqlite.
Complexity: The complexity is linear on the number of pages (dimension) of the source database, beware it can be "slow" if the source database is big, during the copy the source
database is busy and it cannot serve other queries.
Example:
In the example we create a database, we create a table, and then we pushed few rows into the table.
We then create another database, but this time we didn't create the table, neither pushed any row.
As expected, trying to query the second database returned an error.
After copying the content of the first database into the second, the second database has become a perfect copy of the first one.
See also:
ZEESQL.INDEX
The index command accepts 3 different options:
NEW
LIST
DELETE
We will illustrate them separately.
ZEESQL.INDEX NEW
Creates a new secondary index table for the Redis hashes.
The secondary index will refer to the table table_name
and will use the columns indicated in the SCHEMA
parameter.
column_type
can be whatever is accepted by SQLite as column name, suggestions are TEXT
, INT
, FLOAT
or BLOB
.
If a prefix is provided, only the Redis hashes that start with that prefix are indexed in the table. If the prefix is omitted, the *
prefix (catch-all) is assumed.
If the table does not exists when the index is created, zeeSQL
creates it.
If the table already exists, zeeSQL
takes no action. However, if the table exists but contains the wrong columns, zeeSQL
may find it impossible to insert the hashes into the secondary index.
It is possible to create multiple indexes, with the same table, same schema, but different prefix.
zeeSQL
store in the secondary index table, the main Redis hash key, as primary key.
The table created by zeeSQL
behaves like any other table, so it can be queried, modified, and indexed, using the standard ZEESQL.EXEC
interface.
No steps are taken to avoid manual deletions or updates of the secondary index table by the user.
Secondary indexes are univocally identified by the combination of the table in which they write and the prefix.
Example:
In this example, we can see how to create a secondary index, and how the secondary index automatically keeps the values between Redis and zeeSQL in synchronism.
ZEESQL.INDEX LIST
The list option shows the active secondary index.
The secondary indexes are identified by the table in which they write and by the prefix they use to filter the keys.
Example:
ZEESQL.INDEX DELETE
The DELETE
option removes a secondary index.
Hashes that match the prefix are not inserted anymore in the table after the index is removed.
The DELETE
option takes as input the table and the prefix that identifies the secondary index to remove.
If the prefix is omitted the *
(catch-all) prefix is assumed.
Example:
In the example, we first create an index.
Then we add a Redis hash that matches the secondary index prefix, so it is added to the secondary index table.
Then, delete the index.
And we can confirm that new users are not added any more to the secondary index table.
ZEESQL.LICENSE
The SET
option will set the license to use for zeeSQL
.
The license is first checked against our backend server, and if the license is correct, it will return OK
otherwise it will return an error.
It is required an internet connection to set the license.
The SHOW
option will show the license that is actually in use in your zeeSQL
process.
Last updated