Forem: Richard Andersen

Unlock Aerospike Cloud: Exploring the power of the Aerospike LINQPad driver

Richard Andersen — Wed, 28 Feb 2024 02:05:46 +0000

This blog is part of a continuing series showing how to use the Aerospike LINQPad driver. For other blogs in this series, please see:

The Aerospike LINQPad driver is now able to connect to the Aerospike Cloud, providing all the power of LINQPad combined with the Aerospike LINQPad driver.

This includes a graphical interface, LINQ support, and enhanced driver features to support the Aerospike Database. Find more information about LINQPad here.

Let’s look at the new connection dialog and some of the features of the LINQPad driver.

Connection dialog

The new connection dialog will now have two tabs. The first tab will be for native, “self-managed”, Aerospike clusters, and the second tab is used to connect to Aerospike Cloud.

Below shows the connection properties:

Aerospike Cloud connection dialog

A review of each property is below:

The host name is provided in the cloud dashboard. The hyperlink will take you to the Aerospike login screen or, if you are already logged in, to the dashboard.
The cloud client connection port.
If a VPC (AWS private link) is defined, this would be the host name displayed on the cloud dashboard. The host name field would be the VPC endpoint.
The API key is created via the dashboard.
If the API key and secret were exported from the dashboard to your local machine, this button will allow you to import that API Key CSV file. Properties “API Key” (item 4) and “API Secret” (item 7) are not required.
If enabled, the built-in LINQPad password manager is used. When enabled, a dropdown will be displayed to select the Password Name you defined in the manager. The Password Manager can be found on the File menu of LINQPad.
The associated API Key’s secret.
If checked, it will show the API secret in plain text.
The associated cloud namespace.
If provided, a list of set names separated by a comma or space. The set names will be used to populate the sets under the namespace in the LINQPad connection tree (see below image). Also, set and bin “detection” will be performed to obtain the bins and data types. Regardless of whether this field is provided, you can always obtain this information from the “Null Set.” See the “Using Null Set” section below.
Hyperlinks to additional topics.
The timeout values that will be used when obtaining the connection or performing an operation. Note that the “Sleep” field is ignored for cloud connections.

Below shows the relationship between the “set names” connection property and the LINQPad connection tree:

Association between the “Set Names” property and the sets populated in the connection tree

Driver overview

All the features of the Aerospike LINQPad driver are available, including the following:

Graphical interface with all LINQPad features
Drag-and-drop of set names
LINQ
Serialize and deserialize any C# object via the Object-Mapper (POCO)
Auto-Values, which provide dynamic data type conversion from Aerospike types to .Net types without testing or casting
JSON support
Explore the Aerospike API directly or use the driver’s enhanced API

Demonstrates some of the capabilities of LINQPad and the LINQPad driver

Using the Aerospike null set

If the “Set Names” connection property is not provided, the set and bin names will not be provided in the LINQPad connection tree (see image above).
You can still perform queries and API calls using the driver’s API features. Below is an example of a driver API call to obtain the records for Aerospike set “Artist”:

aerospike_cloud.NullSet.Where(ns => ns.Aerospike.SetName == "Artist")

Below is the output:

Result using the driver’s extended API for set “Artist”

You can always extend the “where” clause or interact with the result set.

Samples

You can find samples for Aerospike LINQPad Cloud under the samples folder or under the “Samples” tab in LINQPad.

Review

In this installment of the Aerospike LINQPad driver blog series, we learned how to use LINQPad to connect to Aerospike Cloud. This connection makes exploring your data in the cloud just as easy as other deployment options. Stay tuned for more articles in this series highlighting additional features of the driver.

Using auto-values in Aerospike LINQPad driver

Richard Andersen — Thu, 08 Feb 2024 20:03:53 +0000

When working with NoSQL databases like Aerospike, which are designed for unstructured data storage, adopting strongly typed languages like C# may introduce programming complexities. These can stem from verifying the presence of "bins" (akin to columns) within a record and bin data types that can change between records.

Auto-Values (AValue is the class) simplify the need to check, cast, or convert values between Aerospike data types and .NET data types. They provide a rich set of operations and seamlessly work with any type of data, including collections and JSON. They provide protection against null reference exceptions, invalid cast exceptions, conversion exceptions, etc.
To explore the LINQPad or the Aerospike LINQPad driver in more detail, kindly click on the respective links provided.

Example set

The following illustrates an Aerospike set where different records have a varied number of bins, and bins in the record set can have varying data types (e.g., bin BinA can be a double, long, string, or list in different records).

Display of the DemoTypes set

Note that the LINQPad connection tree (left side) is expanded, illustrating the bins for the set DemoTypes. Under the set name, each bin is listed, displaying its associated data type. Take note of the symbols after the data type. A * indicates this bin has different data types between records. A ? indicates this bin wasn’t present in some of the records within this set. These symbols provide a quick view of the set and expected values.
In the Results pane (bottom-right), bins with null are not present in that record.

Simple cast examples

Below are examples that illustrate the difference between using Aerospike data types, C# data types, and Auto-Values.
Let’s begin with an example where Auto-Values are not used:

Shows that casting to the proper data type is required

Notice that there are no results from the query, even though it should have matched one record. The reason it failed is because “456” is an Int32, and the DB numeric values are Int64. So, “456” needs to be cast to Int64 (long). Also, we need to check to make sure the bin is present in all records; otherwise, a null exception is thrown.

Below is an example showing these changes:

Updated query using required cast

Let’s look at the same query using Auto-Values:

Using Auto-Values doesn’t require testing or casting

In the given example, you do not have to verify the existence of the bin or explicitly cast "456" to a long data type.
Auto-Values take care of not only casting but also converting. Conversion between any of the .NET primitive types to or from any Aerospike data types is seamless. It also handles nullable values, JSON objects, and collection data types (CDT).

Auto-Values support all standard operations like ==, >, <, etc. They also support all the equality and comparison operations.

Below are some additional examples:

Queries above show different uses of Auto-Values

Handling unsupported data types

Aerospike supports a limited set of data types. They include:

String
Integer (Int64)
Double
Boolean
Blob/Bytes (byte[])
List
Map (Directory)
Geospatial (GeoJSON.NET)

The driver has extended support by allowing the following .NET data types:

DateTime, DateTimeOffset, TimeSpan: These values are mapped to either an Aerospike string or long data types depending on the connection or API configuration. Auto-Values will automatically take care of these conversions to or from the database.
JSON: These values are mapped to an Aerospike Map data type where the JSON property name is the key in the Map, and the JSON value is the Map value. JSON arrays are mapped to Aerospike “List.” Nested JSON structures are completely supported, and the driver takes care of all conversions to or from the database.

Below are examples where .NET DateTimeOffset, DateTime, and TimeSpan instances are used to obtain matching records. The values stored in the Aerospike database are either strings or an Int64 value (number of nanoseconds from Unix Epoch). The driver will manage the conversion between these different types.

Queries using .NET DateTime, DateTimeOffset, and Timespan

By the way, you can control how values are converted and displayed through the properties located in the "Display/Conversion Options" section of the connection dialog.

Conversion

Auto-Values provide a rich set of conversion functions, including:

Convert<T> - Will try to convert to the provided .NET data type. If it cannot, an invalid cast exception is thrown.
Is{data type} – Use to test if the underlying .NET data type is that {date type}. Examples of these functions are: IsList, IsCDT, IsJSON, IsInt16, IsNumeric, IsString, etc.
To{data type} – Will try to convert to the .NET {data type}, if possible. Note {data type} can be Dictionary, List, .NET native type, etc. Examples of these functions are: ToList, ToBoolean, ToByte, etc.
TryGetValue<T> – This will try to match a provided value and convert the value into the provided .NET data type. If not successful, the default value of that data type and/or false is returned, depending on usage. This function can be applied against CDTs or non-CDT values.
TryGetValue – This will try to match the provided value. If successful, the matched value is returned as an Auto-Value. If not, an empty Auto-Value, false, or null can be returned depending on usage. This function can be applied against CDTs or non-CDT values.
AsEnumerable – This will convert an Auto-Value into an enumerable object if it is a database CDT. If not a CDT, an empty enumeration is returned. All elements in the CDT are scanned and converted into Auto-Values. This will provide the highest level of protection against invalid casts or null value reference exceptions. This also allows for using the advanced Auto-Value functions outlined in the Collection Data Types section.
Implicit Casting – Auto-Values know how to implicitly cast from an Aerospike data type to any .NET primary type without explicitly providing the type. Detailed documentation can be found through IntelliSense or by reviewing the function’s respective documentation.

Collection data types

Auto-Values can seamlessly be used to find elements within CDTs. These functions are based on standard C# methods. Examples of these operations are:

Contains – Returns true if the matching value is contained in a bin’s value or an element within a CDT. The matching options determine how the matches occur.
FindAll – Returns a collection of matching Auto-Values. A match can occur as a bin’s value or an element within a CDT. The matching options determine how the matches occur. If no matches are found, an empty enumerable is returned.
TryGetValue – Returns the first matching value (as an Auto-Value) contained in a bin’s value or an element within a CDT. If the Auto-Value is not found, an Empty Auto-Value is returned.
OfType<T> - Tries to cast the Auto-Value to the provided .NET type, creating a new collection of those types. If a value cannot be cast, it will be ignored.
Cast<T> -- Will cast the Auto-Value to the provided .NET type. If it cannot be cast, an invalid cast exception will occur.
Convert<T> - Will try to convert the Auto-Value, resulting in a collection of converted .NET values. If an Auto-Value cannot be converted, it will be ignored.

Below are some examples using “Contains” and “FindAll” methods:

Queries using the Contains and FindAll methods

You can find additional examples in the LINQPad sample folder.
Below is an example of “drilling” into several different sub-collections using LINQ. This example finds all invoices for a certain song track, resulting in a collection of customers. This example is located in the CDT-Json-Docs sample LINQPad script.

LINQ Query

Primary key

Auto-Values also extend to primary keys. They have the same features as found in the Bin Auto-Values. There are a few additional features, such as digest values.
If the primary key value is not saved (only the digest is used), matching to a value can still occur using Auto-Values.
Below is an example where the digest is only available; note that the actual value must be used.

Query using a string value by means of the database digest

LINQPad driver samples

You can find samples for the Aerospike LINQPad driver in the samples folder or under the “Samples” tab in LINQPad.

Aerospike for LINQPad

Richard Andersen — Thu, 20 Apr 2023 17:35:06 +0000

This blog explores the new Aerospike driver for LINQPad and its available features. Aerospike for LINQPad 7 is a data context dynamic driver for interactively querying and updating an Aerospike database using "LINQPad". LINQPad is a Graphical Development Tool designed for rapid prototyping, data modeling, data mining, drag-and-drop execution, interactive debugging/testing, quickly learn new frameworks/APIs (e.g., Aerospike API ), etc. The Aerospike driver for LINQPad is designed to support all LINQPad features.

Here is a subset of what you can perform using the driver:

Query any Aerospike Set or Secondary Index using any LINQ command (including joins), interactively.
Use the driver’s extension methods to perform operations like Aerospike Expression , CRUD operations, etc. without understanding the underlying Aerospike API.
Serialize and deserialize any C# object via the Object-Mapper (POCO). The driver supports all C# data types, nested classes, and collections.
Full JSON support using Json.NET.
Execute UDFs directly and display their underlying code. UDFs are treated like C# methods with intellisense and code completion.
Export or import sets directly or with an Aerospike Filter .
Provides metadata about the cluster, including active/inactive nodes, Aerospike server version, etc.
Use the Aerospike API directly to perform advance operations or instantly test snippets used in your application code.

The driver can, also, dynamically detect the structure of records within an Aerospike Set resulting in an easy-to-understand view much like a relational table with enhanced capabilities. Some of these capabilities are:

detection of Bins with the same name but have different data types between records within a Set
records with different Bin structures within a Set
implicit data type conversion without the need to cast or check a Bin's data type for quick data operations
enhanced Aerospike CDT handling
driver extension methods to programmatically interrogate Namespaces, Sets, records, Bins, Bin data types, etc.

The screenshot below show how Aerospike Sets and Bins are represented in LinqPad:

Aerospike Namespaces, Sets, Bins, etc. Representation in LINQPad

The LINQPad connection pane displays the different Aerospike components in an hierarchical manner where namespace is under Aerospike cluster connection. Aerospike sets are under namespaces, and bins are under sets. Below screenshot shows the relationship between these components:

You can drag-and-drop each component onto the LINQPad Query pane to be executed by LINQPad. The execution behavior depends on the component. For example, a set or secondary index will present the records within that component. For other components, the properties are displayed. In all cases, you can always execute the driver’s extension methods. These extension methods, greatly simplify Aerospike API commands like get, put, query, operate, etc. You can also import or export data. Of course, you can always use LINQ against Aerospike sets or secondary indexes. An example of some of the driver extensions follows:

IntelliSense Showing Some of the Extension Methods and Properties on a Set Object

For detailed information about the Aerospike LINQPad driver, see the ReadMe file in GitHub .

Installation

You can install the Aerospike Driver directly from the LINQPad NuGet Manager or from the GitHub Repo.

LINQPad NuGet Manager

Open LINQPad.
Click Add Connection Link.
Select View more drivers….
Select Show all drivers, and then type Aerospike.
Click Install.

Manual (GitHub)

Obtain the latest driver (.LPX6 file) from the Driver folder in GitHub and download it to your computer.
Open LINQPad.
Click Add Connection Link.
Click View more drivers…
Click Install driver from .LPX6 file… and select downloaded lpx6 file.