Forem: Kamlesh Kumar

CSV File Format Guide for Cloud Express: Managing Nodes and Edges

Kamlesh Kumar — Thu, 31 Aug 2023 15:34:53 +0000

This guide is part of a series on Cloud Express – the web application that revolutionizes how you interact with Apache Age. If you haven't already, check out our previous blog for an introduction to Cloud Express and its user-friendly interface.

In Cloud Express, managing your data becomes a breeze, especially with the ability to upload CSV files containing nodes and edges. These CSV files play a pivotal role in constructing and visualizing your projects seamlessly. When you log in using your existing account or Google credentials, you'll step into a world where data organization and project creation are at your fingertips.

Creating Projects and Uploading CSV Files
Upon login, you'll be directed to a dedicated page where the magic happens. Here, you can initiate projects and start uploading CSV files containing both nodes and edges. The beauty lies in the structure and naming conventions of these files.

Distinguishing Between Nodes and Edges
Cloud Express effortlessly distinguishes between nodes and edges by analyzing the filenames. If your file's name contains 'eg_', it's recognized as an edge file. On the other hand, if it carries 'vt_', it's deemed a vertex (node) file. This intuitive separation streamlines your project setup.

Predefined Sample Projects
To make your journey even smoother, Cloud Express offers three predefined project templates you can choose from:

Car Specification
Cybersecurity
p2p (Graphs with Crypto-graphs)

These templates come preconfigured, granting you a head start in your project creation process. You can dive right into these sample projects and tailor them to your specific needs.

Understanding CSV File Formats for Nodes and Edges
Now, let's explore the structure of the CSV files you'll be uploading.

CSV Format for Nodes (Vertices)

Field Name	Field Description
id	Unique identifier for the node (optional if id_field_exists is set to false)
Properties	Columns with node properties (header row with property names)

CSV Format for Edges

Field Name	Field Description
start_id	Node ID where the edge originates
start_vertex_type	Class of the starting node
end_id	Node ID where the edge concludes
end_vertex_type	Class of the ending node
properties	Columns with edge properties (header row with property names)

Access Sample CSV Formats
For a hands-on experience, check out our Sample CSV Format that aligns with these guidelines. This will give you a tangible idea of how to structure your own CSV files for nodes and edges.

Clearing Up the CSV Format
In summary, when crafting your CSV files, ensure that the structure aligns with the guidelines provided above. For nodes, include an id column and subsequent property columns. For edges, list the start_id, start_vertex_type, end_id, end_vertex_type, and the edge properties.

Embrace the simplicity of CSV files in Cloud Express – they're your key to organizing nodes, defining edges, and bringing your projects to life. Whether you're an experienced data manager or just starting out, Cloud Express empowers you to effortlessly manage, visualize, and explore your data.

Introducing Cloud-express: Access Apache Age with ease on Browser

Kamlesh Kumar — Thu, 31 Aug 2023 15:18:38 +0000

Introducing Cloud-express: Access Apache Age with Ease on Browser

Cloud-express is an exciting web application that brings Apache Age, an advanced database management system, right to your fingertips. This innovative tool bridges the gap between users and Apache Age, making database management more accessible than ever before. With Cloud-express, you can now harness the power of Apache Age through a user-friendly web interface, eliminating the need for offline access and complex setups.

Simplified Access
Gone are the days of intricate installations and offline access struggles. Cloud-express offers a seamless experience by providing direct access to Apache Age from your web browser. No more wrestling with software installations – Cloud-express simplifies your journey by enabling you to utilize Apache Age's potent capabilities online.

Key Features

User-Friendly Interface: Cloud-express welcomes users with an intuitive interface, making database management a breeze, even for newcomers.
Web-Based Access: Say farewell to offline limitations. Access and manage your databases through Cloud-express from anywhere with an internet connection.
Effortless Setup: With Cloud-express, there's no need to be a tech guru. The setup process is straightforward, getting you up and running in no time.
Advanced Capabilities: Leverage Apache Age's cutting-edge features directly from Cloud-express, enhancing your database management efficiency.

Getting Started

Node Version: Ensure you have Node version ^14.16.0 installed for optimal performance.
Node Module - pm2: For production mode, install the pm2 module with npm i pm2.

Running Cloud-express

Frontend: Install required modules using npm run setup. Run the frontend with npm run start.
Backend: Navigate to the backend folder and use node index.js to start the backend.

Building the Project

Build Front-end: Generate the front-end build with npm run build-front.
Build Back-end: Create the back-end build using npm run build-back.

Production Mode

Stop existing instances: pm2 stop ag-viewer-develop, pm2 delete ag-viewer-develop.
Start in production: pm2 start ecosystem.config.js.

Setting Up PostgreSQL with AGE Extension

Use Docker: Install Docker compatible with your OS. Pull the AGE image with docker pull apache/age.
Create Docker Container: Run docker run --name myPostgresDb -p 5455:5432 -e POSTGRES_USER=postgresUser -e POSTGRES_PASSWORD=postgresPW -e POSTGRES_DB=postgresDB -d apache/age.
Access PostgreSQL Shell: Use docker exec -it myPostgresDb bash to enter the Docker shell, then psql -U postgresUser postgresDB to access PostgreSQL shell.
Alternatively, access directly: Use psql -U postgresUser -d postgresDB -p 5455 -h localhost and enter postgresPW when prompted.

Connecting to Cloud-express

Server Details: Connect URL - localhost, Port - 5455, Database Name - postgresDB, User Name - postgresUser, Password - postgresPW.

Exploring Projects

Dive into predefined projects or create your own. Enjoy the flexibility to tailor your database management experience.

Experience the Difference
Cloud-express revolutionizes how you interact with Apache Age. Say hello to streamlined access, enhanced capabilities, and a user-centric approach. Whether you're a database enthusiast or a newcomer, Cloud-express empowers you to manage databases effortlessly.

Licensed for Freedom
Just like Apache Age Viewer, Cloud-express operates under the Apache License, Version 2.0. Feel the freedom to explore, innovate, and manage databases with confidence.

Embrace Cloud-express and unlock a new era of simplified database management. Access, manage, and thrive with the power of Apache Age at your fingertips.

Writing Regression Tests for Apache AGE High Availability (AGEHA) or pgpool-II- A Quick Guide

Kamlesh Kumar — Fri, 12 May 2023 20:06:08 +0000

Introduction

One of the key aspects of maintaining robust software is the consistent application of testing, and regression tests play a significant role in this process. They help ensure new changes don't disrupt existing functionality or introduce new bugs. In this blog post, we will guide you on writing regression tests for the AGEHA version of Pgpool-II (version 4.4.2), which is designed to boost the speed and provide high availability for Apache AGE queries.

Before proceeding, if you are unfamiliar with how to run regression tests in Pgpool-II, please refer to our previous blog post: Running Regression Test for AGE-AGEHA (AGE High Availability) and in Pgpool-II version 4.4.2.

Understanding the Regression Test Script

First surface through the code avaliable here.
**
An overview how these regression tests are working.**

To write your own regression tests, it's crucial to understand how the existing test script works. Let's take a look at the structure of a regression test script for Pgpool-II (AGEHA):

The script begins with setting up the test environment, which includes sourcing necessary libraries and setting up directories. Then, the test environment for Pgpool-II is created with the pgpool_setup command. After the setup, the script starts the Pgpool-II servers and sets up the necessary environment variables.

Next, the script creates a new graph database in each PostgreSQL backend and initiates some data. After that, it executes various Cypher queries to interact with the database.

The script then checks the pgpool logs to ensure the queries have been executed on the correct backend nodes, depending on the type of the query (read or write) and the weights of the backend nodes. If the queries haven't been executed as expected, the script exits with a failure message. Otherwise, it continues to the next set of queries.

Finally, the script shuts down all servers and exits with a success message.

Example: Writing a Regression Test for Simple Load Balancing

In this example, we will explain the process of writing a regression test for simple load balancing using Cypher queries. We will follow the pattern shown in the provided code snippet.

The test is designed to check whether the load balancing works as expected when executing Cypher queries. Here's a breakdown of the test steps:

Execute Cypher queries to create vertices: The following two queries are executed to create two vertices with different properties in the graph.

Follwoing code is taken from here.

$PSQL template1 <<EOF
SELECT * FROM cypher('test_graph', \$\$ CREATE (:vertex1 {i: 123}) \$\$) as (v agtype);
SELECT * FROM cypher('test_graph', \$\$ CREATE (:vertex2 {i: 124}) \$\$) as (v agtype);
EOF

Check if simple load balancing worked: After executing the queries, the script checks the pgpool logs to ensure that the queries were executed on the correct backend node (in this case, DB node id: 1). This is done using the fgrep command, which searches for the expected log message in the log/pgpool.log file. If the message is found, the test proceeds; otherwise, the script exits with a failure message.

fgrep "SELECT * FROM cypher('test_graph', \$\$ MATCH (v) RETURN v \$\$) as (v agtype);" log/pgpool.log |grep "DB node id: 1">/dev/null 2>&1
if [ $? != 0 ];then
# expected result not found
    echo fail: select is sent to zero-weight node.
    ./shutdownall
    exit 1
fi
echo ok: simple load balance works.

This example demonstrates a simple regression test for load balancing. You can follow this pattern to create your own tests by modifying the Cypher queries and the expected log messages.

About the .sh File

The .sh file is a shell script file that contains a series of commands to be executed by the shell interpreter. In the context of writing regression tests, the .sh file contains the test script, which sets up the test environment, executes the Cypher queries, checks the results, and cleans up the test environment after completion.

Writing Your Own Regression Test

Now that you understand the structure of an existing regression test, let's explore how you can write your own. Here are some steps to get started:

Set Up the Test Environment: At the beginning of your script, you should set up the test environment similar to the one in the existing test script.
Create Your Test Data: Depending on the specific feature or functionality you're testing, you may need to create your own test data. This could involve creating a new graph database and populating it with data, or modifying existing data.
Write Your Test Queries: The core part of your regression test will be the Cypher queries you use to interact with the database. Try to cover as many edge cases as possible to ensure the feature or functionality is robust.
Check the Results: After executing your test queries, you should check whether the queries were executed as expected. You can do this by checking the pgpool logs, as in the existing test script, or by querying the database and comparing the results to the expected output.
Clean Up: Finally, clean up the test environment by shutting down all servers and removing any temporary files or directories.

Remember, the key to writing a good regression test is to cover as many edge cases as possible and to check the results thoroughly to ensure the system behaves as expected. Happy testing!

Further references:
Apache-age AGEHA(Pgpool-II) repo: https://github.com/apache/age/tree/AGEHA
Apache-age-repo: https://github.com/apache/age/
Apache-age documentation: https://age.apache.org/age-manual/master/intro/overview.html

Running Regression Test for AGE-AGEHA (AGE High Availability) and in Pgpool-II version 4.4.2.

Kamlesh Kumar — Thu, 11 May 2023 14:30:19 +0000

Introduction

Apache AGE has brought about an innovative feature to the table - Pgpool-II, which is specially designed to boost the speed and provide high availability for Apache AGE queries. This feature is succinctly named AGEHA, an abbreviation for AGE High Availability. One of the key aspects of maintaining the robustness and reliability of AGEHA is regression testing. This blog post will guide you through the process of running regression tests in the AGEHA or Pgpool-II (version 4.4.2), along with the prerequisites required.

Prerequisites

Before diving into the regression testing process, let's review the prerequisites and how to install them on Ubuntu.

PostgreSQL: Pgpool-II requires PostgreSQL for operation.

sudo apt update
sudo apt install postgresql postgresql-contrib

Perl: Pgpool-II uses Perl for running its regression tests.

sudo apt install perl

Git: Git is required to clone the source code.

sudo apt install git

Cloning and Installing AGEHA/Pgpool-II

With the prerequisites in place, we can now clone and install AGEHA/Pgpool-II. Use the following commands:

git clone https://github.com/apache/age -b AGEHA 
cd age

Compile and install Pgpool-II with:

./configure
make
sudo make install

Running Regression Test in Pgpool-II

Having installed AGEHA/Pgpool-II, we can now proceed to the regression test. Regression testing checks the system's behavior and helps identify potential bugs or inconsistencies. Here's how you can perform the regression test in Pgpool-II:

Navigate to the src/test/regression directory in the cloned repository.

cd src/test/regression

Execute the pgpool_setup script, which creates a fresh test environment.

./pgpool_setup

Run the regression tests with make check. This command runs a suite of regression tests to ensure the system behaves as expected after new changes or updates. "make check" command is designed to run all suite of regression tests.

make check

Running Regression Test for Cypher Queries individualy.

In addition to the general regression tests, AGEHA also supports regression tests specifically for Cypher queries. To execute these tests:

Navigate to the 01.Cypher_load_balance directory.

01.

cd tests/Cypher_load_balance

Run the test.sh script.

./test.sh

This script will execute the regression test specifically for Cypher queries.

Conclusion

Regression testing plays a vital role in ensuring that the system behaves as expected, and newly introduced changes do not disrupt the existing functionality. In the context of AGEHA/Pgpool-II for Apache AGE, regression testing contributes significantly to maintaining its high-speed and high-availability features. We hope this guide has provided you with a clear understanding of how to run regression tests in (AGEHA)/Pgpool-II version 4.4.2 . Happy testing!

Apache-age: A Powerful and Open Source Graph Database Solution. [Part # 2]

Kamlesh Kumar — Sun, 19 Mar 2023 19:38:10 +0000

Querying the Data

Welcome back to our series on building a social network app with apche-age! In our previous blog post, we covered the basics of data modeling, setting up Neo4j, creating the graph, and adding sample data. Now that we have our graph populated with users, posts, comments, and relationships, it's time to start querying the data. In this post, we'll explore some example queries for retrieving useful information from our social network app. If you haven't read our previous post yet, we recommend starting there to get caught up on the basics of building a social network app with apche-age here

Query to get all posts

SELECT * from cypher('social_network', $$

        MATCH (p:Post)
        RETURN p.title, p.content

$$) as (V agtype, C agtype);

This query returns the title and content of all posts in the graph.

Get all posts and their comments

SELECT * from cypher('social_network', $$
        MATCH (V)-[R:COMMENTED]-(V2)
        RETURN V,R,V2
$$) as (V agtype, R agtype, V2 agtype);

This query returns the title of each post and the content of each comment on that post.

Get all posts and their likes

SELECT * from cypher('social_network', $$
       MATCH (p:Post)<-[:LIKED]-(u:User)
       RETURN p.title, COUNT(u) as likes
$$) as (V agtype, R agtype);

This query returns the title of each post and the number of likes it has.

Get all posts and their authors

SELECT * from cypher('social_network', $$

    MATCH (u:User)-[:POSTED]->(p:Post)
    RETURN p.title, u.name

$$) as (V agtype, R agtype);

This query returns the title of each post and the name of its author.

Get all users and the users they follow

SELECT * from cypher('social_network', $$
       MATCH (u:User)-[:FOLLOWS]->(f:User)
       RETURN u.name, COLLECT(f.name) as following
$$) as (V agtype, R agtype);

This query returns the name of each user and a list of the names of the users they follow.

References

For further help you can look though the documentation here and Apache-age repo here.

"Building a Social Network App with Apache-age: A Beginner's Guide"

Kamlesh Kumar — Sun, 19 Mar 2023 19:30:24 +0000

Introduction

Apache-age is a popular graph database that is widely used for building applications that deal with highly interconnected data. In this blog post, we'll walk you through building a social network app using apache-age. We'll cover the data model, setting up apache-age, creating the graph, adding sample data, and querying the data.

The Data Model

A social network app has many interconnected entities, such as users, posts, comments, and likes. In a graph database like apache-age, we can represent each of these entities as a node, and the relationships between them as edges. Here's an example data model:

User: A user of the app
Post: A post made by a user
Comment: A comment made on a post
Like: A like given to a post

We'll also have the following relationships between entities:

FOLLOWS: A user follows another user
POSTED_BY: A post is made by a user
COMMENTED_ON: A comment is made on a post
LIKED: A user likes a post

Here's a diagram of the data model:

┌─────┐ FOLLOWS ┌─────┐
│User │────────>│User │
└─────┘         └─────┘
    |             |
    | POSTED_BY   |
    v             v
┌─────┐       ┌─────┐
│Post │       │User │
└─────┘       └─────┘
    |             |
    | COMMENTED_ON|
    v             v
┌───────┐     ┌─────┐
│Comment│     │Post │
└───────┘     └─────┘
    |             |
    | LIKED       |
    v             v
┌─────┐       ┌─────┐
│User │       │Post │
└─────┘       └─────┘

In this data model, each node represents an entity in the social network, and each edge represents a relationship between them. For example, a User node might be connected to a Post node by a POSTED edge, indicating that the user created the post. Similarly, a User node might be connected to another User node by a FOLLOWS edge, indicating that the user is following the other user.

Setting up Apache Age

Before we can start building our social network app, we need to set up apache-age. Using the blog at link. Once installed, we can start the apache-age server and open the apache-age viewer in browser.

Creating the Graph

Once we have apache-age set up, we can create the graph using Cypher

select create_graph('social_network');

Here's some Cypher code to create the nodes and edges in our social network graph:

SELECT * from cypher('social_network', $$
        CREATE (Alice:User {name: 'Alice'})
CREATE (Bob:User {name: 'Bob'})
CREATE (Charlie:User {name: 'Charlie'})
CREATE (Dave:User {name: 'Dave'})

CREATE (Post1:Post {title: 'My First Post', content: 'Hello world!'})
CREATE (Post2:Post {title: 'My Second Post', content: 'This is my second post.'})
CREATE (Post3:Post {title: 'My Third Post', content: 'Just testing things out.'})

CREATE (Comment1:Comment {content: 'Nice post!'})
CREATE (Comment2:Comment {content: 'I agree.'})
CREATE (Comment3:Comment {content: 'Could use some work.'})

CREATE (Alice)-[:POSTED]->(Post1)
CREATE (Bob)-[:POSTED]->(Post2)
CREATE (Alice)-[:POSTED]->(Post3)

CREATE (Charlie)-[:COMMENTED]->(Comment1)
CREATE (Dave)-[:COMMENTED]->(Comment2)
CREATE (Bob)-[:COMMENTED]->(Comment3)

CREATE (Alice)-[:LIKED]->(Post2)
CREATE (Bob)-[:LIKED]->(Post1)
CREATE (Charlie)-[:LIKED]->(Post2)
CREATE (Dave)-[:LIKED]->(Post3)

CREATE (Alice)-[:FOLLOWS]->(Bob)
CREATE (Alice)-[:FOLLOWS]->(Charlie)
CREATE (Bob)-[:FOLLOWS]->(Charlie)
CREATE (Charlie)-[:FOLLOWS]->(Dave)


$$) as (V agtype);

This code creates nodes for each user and post, comment, and edges to represent the relationships between them. For example, the CREATE (Alice)-[:POSTED]->(Post1) statement creates a POSTED relationship between the Alice node and the Post1 node.

Adding more Sample Data

Now that we have created our graph, we can add some sample data to it. Here's some Cypher code to add sample data:

SELECT * from cypher('social_network', $$

        CREATE (Alice)-[:POSTED]->(:Post {title: 'Hello World', content: 'My first post!'})
        CREATE (Bob)-[:POSTED]->(:Post {title: 'Second Post', content: 'This is my second post.'})
        CREATE (Charlie)-[:POSTED]->(:Post {title: 'Third Post', content: 'Just testing things out.'})
        CREATE (Dave)-[:POSTED]->(:Post {title: 'Fourth Post', content: 'Hello everyone!'})

        CREATE (Charlie)-[:COMMENTED]->(:Comment {content: 'Nice post!'})
        CREATE (Dave)-[:COMMENTED]->(:Comment {content: 'I agree.'})
        CREATE (Bob)-[:COMMENTED]->(:Comment {content: 'Could use some work.'})

        CREATE (Alice)-[:LIKED]->(:Post {title: 'Second Post'})
        CREATE (Bob)-[:LIKED]->(:Post {title: 'Hello World'})
        CREATE (Charlie)-[:LIKED]->(:Post {title: 'Second Post'})
        CREATE (Dave)-[:LIKED]->(:Post {title: 'Fourth Post'})

        CREATE (Alice)-[:FOLLOWS]->(Bob)
        CREATE (Alice)-[:FOLLOWS]->(Charlie)
        CREATE (Bob)-[:FOLLOWS]->(Charlie)
        CREATE (Charlie)-[:FOLLOWS]->(Dave)

$$) as (V agtype);

This code adds more posts, comments, and edges to the graph.

You can query to see the data:

SELECT * from cypher('social_network', $$
        MATCH (V)-[R]-(V2)
        RETURN V,R,V2
$$) as (V agtype, R agtype, V2 agtype);

So at this point we have completed our data creation, now our model is ready and we can query the data as per our requirements.

To learn more about querying the data you can follow the part 2 of the blog at link.

References
For further help you can look though the documentation here and Apache-age repo here.

Apache-age: The best Open Source Graph Database Solution

Kamlesh Kumar — Wed, 08 Mar 2023 13:30:38 +0000

Are you looking for powerful and open-source Graph database solution?

Graph databases are becoming increasingly popular due to their ability to store and manage complex relationships between data points. They are commonly used in applications such as social networks, recommendation systems, and fraud detection. However, finding the right graph database solution can be a daunting task. There are some popular graph databases on the market, but they can be expensive to use. That's where "apache-age" comes in - an open source, PostgreSQL-based graph database solution that offers a lot of the same functionality as the other graph databases do, but at no cost.

One of the key advantages of "apache-age" is its basis on PostgreSQL, which is one of the most powerful and widely used open source relational databases. This means that "apache-age" can leverage all of the features and benefits of PostgreSQL, including its robust transactional support, data integrity, and scalability. Additionally, "apache-age" is able to take advantage of PostgreSQL's powerful query optimization engine, which allows it to execute complex queries quickly and efficiently.

One of the advantages of "apache-age" is its compatibility with the open Cypher Query Language, which is a widely used and standardized language for querying graph databases and easy to use. which means that developers can use it to easily to interact with their graph database. Additionally, because Cypher is an open standard, developers can use it to query other graph databases as well, providing greater flexibility and interoperability.

The "apache-age" extension also includes a number of features that make it easier to work with graph data. For example, it includes a number of built-in functions that allow you to easily traverse relationships between nodes, find shortest paths, and perform other common graph operations.

One of the most compelling aspects of "apache-age" is that it is completely open source and is backed by the Apache Software Foundation. This means that it is free to use, and that it is actively maintained and supported by a community of developers. It also means that you have full control over the source code, and can customize it to suit your specific needs.

"Apache-age" is continuously improving with new features such as drivers for multiple programming languages like Golang, high availability support through pgpool-II, and upcoming desktop management applications. These improvements will make it even easier to integrate, work with and visualize your graph data. The future looks bright for "apache-age" with continued development and innovation on the horizon.

Installation and documentation for "apache-age" is straightforward and well-documented. You can find detailed instructions on how to install and use the extension at this blog.

The offical documentation of apche-age includes a number of tutorials and examples that can help you get started with using "apache-age" in your own projects.

In conclusion, if you're looking for a powerful and open source graph database solution, "apache-age" is definitely worth considering. It offers a lot of the functionality, but at no cost, and it leverages the power of PostgreSQL to provide a scalable and efficient graph database. With its support for open-cypher and its built-in graph functions, it is also easy to work with and customize to suit your needs. Give it a try today and see how it can improve your graph database workflows.

I hope this blog post helps to persuade you of the benefits of using "apache-age" as a graph database solution, and provides a useful introduction to its features and advantages. Don't hesitate to check out the official documentation and GitHub repository for more information and resources.

For more details visit apache-age official website: