Forem: Dinesh

A real-time tech course finder, created using Elasticsearch, Python, React+Redux, Docker, and Kubernetes.

Dinesh — Tue, 24 Aug 2021 18:36:58 +0000

Github repo: https://github.com/dineshsonachalam/tech-courses-search-engine

Demo

What's this project all about?

This project showcases how to build real-time search engines like Google, Coursera, Medium, etc. We focus on the following aspects as part of this project.

1. Understanding all significant components in ElasticSearch and its Auto completion feature.
2. Building an API service that interacts with ElasticSearch to be used by the UI.
3. Testing API using Pytest.
4. Building UI using React and Redux.
5. Testing UI using Cypress.

Application Architecture

1. Understanding all significant components in ElasticSearch and it's Auto completion feature.

What is ElasticSearch?

Free and Open, Distributed, RESTful Search Engine. You can use Elasticsearch to store, search, and manage data for:

Logs
Metrics
A search backend
Application monitoring
Endpoint security

How does Elasticsearch work?

Let's understand some basic components of how it organizes data in ElasticSearch.

Logical components

Documents:

Documents are the low level unit of information that can be indexed in Elasticsearch expressed in JSON, which is the global internet data interchange format. You can think of a document like a row in a relational database, representing a given entity — the thing you’re searching for. In Elasticsearch, a document can be more than just text, it can be any structured data encoded in JSON. That data can be things like numbers, strings, and dates. Each document has a unique ID and a given data type, which describes what kind of entity the document is. For example, a document can represent an encyclopedia article or log entries from a web server.

Indices:

An index is a collection of documents that have similar characteristics. An index is the highest level entity that you can query against in Elasticsearch. You can think of the index as being similar to a database in a relational database schema. Any documents in an index are typically logically related. In the context of an e-commerce website, for example, you can have an index for Customers, one for Products, one for Orders, and so on. An index is identified by a name that is used to refer to the index while performing indexing, search, update, and delete operations against the documents in it.

Index templates:

An index template is a way to tell Elasticsearch how to configure an index when it is created. The template is applied automatically whenever a new index is created with the matching pattern.

Backend components

Cluster:

An Elasticsearch cluster is a group of one or more node instances that are connected together.

Node:

A node is a single server that is a part of a cluster. A node stores data and participates in the cluster’s indexing and search capabilities. An Elasticsearch node can be configured in different ways:

(i) Master Node — Controls the Elasticsearch cluster and is responsible for all cluster-wide operations like creating/deleting an index and adding/removing nodes.

(ii) Data Node — Stores data and executes data-related operations such as search and aggregation.

(iii) Client Node — Forwards cluster requests to the master node and data-related requests to data nodes.

Shards:

Elasticsearch provides the ability to subdivide the index into multiple pieces called shards. Each shard is in itself a fully-functional and independent “index” that can be hosted on any node within a cluster. By distributing the documents in an index across multiple shards, and distributing those shards across multiple nodes, Elasticsearch can ensure redundancy, which both protects against hardware failures and increases query capacity as nodes are added to a cluster.

Replicas:

Elasticsearch allows you to make one or more copies of your index’s shards which are called replica shards or just replicas.

How to implement Autocompletion ElasticSearch feature?

Start ElasticSearch Docker container

mkdir -p ES_DATA && docker run -v $(pwd)/ES_DATA:/usr/share/elasticsearch/data -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms750m -Xmx750m" -p 9200:9200 elasticsearch:7.12.0

Verify the health status of your cluster.

dineshsonachalam@macbook ~ % curl --location --request GET 'http://elasticsearch:9200/_cat/health'
1629473241 15:27:21 docker-cluster green 1 1 0 0 0 0 0 0 - 100.0%

Create an index template that contains the following properties topic, title, URL, labels, and upvotes.

curl -X PUT "elasticsearch:9200/_index_template/template_1?pretty" -H 'Content-Type: application/json' \
-d'{
    "index_patterns": "cs.stanford",
    "template": {
        "settings": {
            "number_of_shards": 1
        },
        "mappings": {
            "_source": {
                "enabled": true
            },
            "properties": {
                "topic": {
                    "type": "text"
                },
                "title": {
                    "type": "completion"
                },
                "url": {
                    "type": "text"
                },
                "labels": {
                    "type": "text"
                },
                "upvotes": {
                    "type": "integer"
                }
            }
        }
    }
}'

Validate if the index template is available.

dineshsonachalam@macbook ~ % curl --location --request GET 'http://elasticsearch:9200/_index_template/template_1'
{
    "index_templates": [
        {
            "name": "template_1",
            "index_template": {
                "index_patterns": [
                    "cs.stanford"
                ],
                "template": {
                    "settings": {
                        "index": {
                            "number_of_shards": "1"
                        }
                    },
                    "mappings": {
                        "_source": {
                            "enabled": true
                        },
                        "properties": {
                            "upvotes": {
                                "type": "integer"
                            },
                            "topic": {
                                "type": "text"
                            },
                            "title": {
                                "type": "completion"
                            },
                            "url": {
                                "type": "text"
                            },
                            "labels": {
                                "type": "text"
                            }
                        }
                    }
                },
                "composed_of": []
            }
        }
    ]
}

Create a new index called cs.stanford

dineshsonachalam@macbook ~ % curl --location --request PUT 'http://elasticsearch:9200/cs.stanford/'
{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "cs.stanford"
}

Validate if the cs.stanford index is available.

dineshsonachalam@macbook ~ % curl --location --request GET 'http://elasticsearch:9200/cs.stanford/'
{
    "cs.stanford": {
        "aliases": {},
        "mappings": {
            "properties": {
                "labels": {
                    "type": "text"
                },
                "title": {
                    "type": "completion",
                    "analyzer": "simple",
                    "preserve_separators": true,
                    "preserve_position_increments": true,
                    "max_input_length": 50
                },
                "topic": {
                    "type": "text"
                },
                "upvotes": {
                    "type": "integer"
                },
                "url": {
                    "type": "text"
                }
            }
        },
        "settings": {
            "index": {
                "routing": {
                    "allocation": {
                        "include": {
                            "_tier_preference": "data_content"
                        }
                    }
                },
                "number_of_shards": "1",
                "provided_name": "cs.stanford",
                "creation_date": "1629526849180",
                "number_of_replicas": "1",
                "uuid": "NrvQ6juOSNmf0GOPO2QADA",
                "version": {
                    "created": "7120099"
                }
            }
        }
    }
}

Add documents to cs.stanford index.

cd backend && python -c 'from utils.elasticsearch import Elasticsearch; es = Elasticsearch("cs.stanford"); es.add_documents()' && cd ..

Get the total count of the documents in cs.stanford index. We can able to see that the document count is 1350.

dineshsonachalam@macbook tech-courses-search-engine % curl --location --request GET 'http://elasticsearch:9200/cs.stanford/_count'
{
    "count": 1350,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    }
}

Use ElasticSearch suggesters search for autocompletion. The suggest feature suggests similar looking terms based on a provided text by using a suggester.

dineshsonachalam@macbook tech-courses-search-engine % cd backend && python -c 'from utils.filters import SearchFilters; search = SearchFilters("cs.stanford"); print(search.autocomplete(query="python"))' && cd ..
[
    {
        "id": 1,
        "value": "Python Data Science Handbook"
    },
    {
        "id": 2,
        "value": "Python Game Programming Tutorial: SpaceWar"
    },
    {
        "id": 3,
        "value": "Python for Beginners - Learn Python Programming La"
    },
    {
        "id": 4,
        "value": "Python for Data Science and Machine Learning Bootc"
    },
    {
        "id": 5,
        "value": "Python for Security Professionals"
    }
]

2. Building an API service that interacts with ElasticSearch to be used by the UI.

Start the ElasticSearch, Backend and Frontend services

sh dev-startup.sh

API Documentation

ElasticSearch Autocomplete

  GET /autocomplete

Parameter	Type	Description
`query`	`string`	Required. Query string

Sample response

dineshsonachalam@macbook ~ % curl --location --request GET 'elasticsearch:8000/autocomplete?query=python'
[
    {
        "id": 1,
        "value": "Python Data Science Handbook"
    },
    {
        "id": 2,
        "value": "Python GUI with Tkinter Playlist"
    },
    {
        "id": 3,
        "value": "Python Game Programming Tutorial: SpaceWar"
    },
    {
        "id": 4,
        "value": "Python PostgreSQL Tutorial Using Psycopg2"
    },
    {
        "id": 5,
        "value": "Python Programming for the Raspberry Pi"
    }
]

Query Search

  POST /string-query-search

Parameter	Type	Description
`query`	`string`	Required. Query string

Sample response

dineshsonachalam@macbook ~ % curl --location --request POST 'elasticsearch:8000/string-query-search?query=python'
[
    {
        "id": 1,
        "title": "Google's Python Class",
        "topic": "Python",
        "url": "https://developers.google.com/edu/python/",
        "labels": [
            "Free",
            "Python 2"
        ],
        "upvotes": 213
    },
    {
        "id": 2,
        "title": "Complete Python Bootcamp",
        "topic": "Python",
        "url": "https://click.linksynergy.com/deeplink?id=jU79Zysihs4&mid=39197&murl=https://www.udemy.com/complete-python-bootcamp",
        "labels": [
            "Paid",
            "Video",
            "Beginner",
            "Python 3"
        ],
        "upvotes": 196
    },
    {
        "id": 3,
        "title": "Automate the Boring Stuff with Python",
        "topic": "Python",
        "url": "http://automatetheboringstuff.com/",
        "labels": [
            "Free",
            "Book"
        ],
        "upvotes": 93
    },
    {
        "id": 4,
        "title": "Official Python Tutorial",
        "topic": "Python",
        "url": "https://docs.python.org/3/tutorial/index.html",
        "labels": [
            "Free"
        ],
        "upvotes": 74
    },
    {
        "id": 5,
        "title": "Working with Strings in Python",
        "topic": "Python",
        "url": "https://academy.vertabelo.com/course/python-strings",
        "labels": [
            "Free",
            "Beginner",
            "Python 3"
        ],
        "upvotes": 4
    },
    {
        "id": 6,
        "title": "Learn Python the Hard Way",
        "topic": "Python",
        "url": "https://learnpythonthehardway.org/book/",
        "labels": [
            "Paid",
            "Book",
            "Python 3"
        ],
        "upvotes": 293
    },
    {
        "id": 7,
        "title": "Python for Beginners - Learn Python Programming Language in 2 Hours",
        "topic": "Python",
        "url": "https://www.youtube.com/watch?v=yE9v9rt6ziw",
        "labels": [
            "Free",
            "Video",
            "Beginner",
            "Python 3"
        ],
        "upvotes": 62
    },
    {
        "id": 8,
        "title": "Automate the Boring Stuff with Python",
        "topic": "Python",
        "url": "https://click.linksynergy.com/deeplink?id=jU79Zysihs4&mid=39197&murl=https://www.udemy.com/automate/",
        "labels": [
            "Paid",
            "Video",
            "Beginner"
        ],
        "upvotes": 45
    },
    {
        "id": 9,
        "title": "Introduction to Programming with Python",
        "topic": "Python",
        "url": "https://mva.microsoft.com/en-US/training-courses/introduction-to-programming-with-python-8360",
        "labels": [
            "Free",
            "Video"
        ],
        "upvotes": 41
    },
    {
        "id": 10,
        "title": "A Byte of Python",
        "topic": "Python",
        "url": "http://www.swaroopch.com/notes/python/",
        "labels": [
            "Free"
        ],
        "upvotes": 22
    }
]

3. Testing API using Pytest

Pytest is a testing framework based on python. It is mainly used to write API based test cases. Here we are going to test our two API's (autocomplete and string-query-search).

Start Pytest:

dineshsonachalam@macbook tech-courses-search-engine % pytest backend
=========================================== test session starts ===========================================
platform darwin -- Python 3.9.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /Users/dineshsonachalam/Desktop/tech-courses-search-engine
plugins: cov-2.12.1, metadata-1.11.0
collected 2 items                                                                                         

backend/tests/test_api.py ..                                                                        [100%]

============================================ 2 passed in 0.35s ============================================
dineshsonachalam@macbook tech-courses-search-engine %

4. Building UI using React and Redux.

What is React?

A declarative, efficient, and flexible JavaScript library for building user interfaces.

What is Redux?

Redux is a JS library for managing client data in applications. Redux allow your state to be available in one place. It is used to manage data in your application.

Things to care about when using redux:

Identify the state.
Write good reducers.
Let's redux state handle the rest.

Building Parts of redux:

Action -> Action have a type field that tells what kind of action to perform and all other fields contain information or data.
Reducer -> They are functions that take the (current state and action) and return the new state and tell the store how to do.
Store -> The store is the object which holds state of the application.

React components used in our application:

What are React components?

Components are independent and reusable bits of code. They serve the same purpose as JavaScript functions, but work in isolation and return HTML via a render() function.

Components are classified into two types, Class components and Function components.

What's the difference between class vs functional components:

In class component, we can access the value of the state by using this.state inside JSX and we would use setState to update the value of the state. You can set the function inside the event or outside of the render() method -- for readability.

In functional component, we would use useState to assign initial state and we would use setCount (in our example) to update the state. If we want to access the value of the state, we can omit this.state and call the name of the state instead, in our case, it would just be count.

React components used in our application:

Here all our React components are available in the src/components folder.

dineshsonachalam@macbook frontend % tree src/components 
src/components
├── Nav.js
├── ResponsiveAntMenu.js
├── SearchBar.js
└── SearchResults.js

0 directories, 4 files

How Redux is integrated into this React application:

Here all our Redux components are available in the src/redux folder. Here we intialized Actions, Search Reducer and Redux store.

dineshsonachalam@macbook frontend % tree src/redux 
src/redux
├── actionTypes.js
├── actions.js
├── reducers
│   ├── index.js
│   └── searchReducer.js
└── store.js

1 directory, 5 files

To start the UI in development mode:

npm i && npm run start --prefix frontend

5. Testing UI using Cypress.

What is Cypress?

Fast, easy and reliable testing for anything that runs in a browser. Cypress is the most popular choice for Integration testing for web applications.

Cypress Features

Test runner: So hands down one of the best features about Cypress is its test runner. It provides a whole new experience to end-to-end testing.
Setting up tests: Another great feature that we talked about already is setting up tests are extremely easy, you just install Cypress and then everything gets set up for you
Automatic waits – you will barely have to use waits when using Cypress
Stubbing – you can easily stub application function behavior and server response.

Running Cypress Integration test

The cypress integration tests for our application is available at frontend/cypress/integration/search-courses.spec.js filepath.

dineshsonachalam@macbook tech-courses-search-engine % tree frontend/cypress
frontend/cypress
├── fixtures
│   └── example.json
├── integration
│   └── search-courses.spec.js
├── plugins
│   └── index.js
└── support
    ├── commands.js
    └── index.js

4 directories, 5 files
dineshsonachalam@macbook tech-courses-search-engine %

Running your Cypress Test in the Cypress Test Runner:

To open the Cypress Test Runner, you can execute the following command below:

npx cypress open

Once the Cypress Test Runner opens up, you can execute your test which will show results similar to this below:

You can see all the Cypress commands listed below such as visit, URL & title
All your successful assertions will show in Green and failed assertions in Red.

License

MIT © dineshsonachalam

Redis caching layer for Cloudflare KV in Golang

Dinesh — Wed, 18 Aug 2021 16:29:38 +0000

Why redis-cloudflare-kv?

Cloudflare KV is a highly distributed, eventually consistent, key-value store that spans Cloudflare's global edge. It allows you to store billions of key-value pairs and read them with some average latency (2 to 3 seconds) anywhere in the world.
Cloudflare KV Read operation is very slow when compared to Redis which offers memory efficiency, fast operating speed, and high availability. So it's better to use Redis as a caching layer for Cloudflare KV to provide fast, economic scaling for your application.
redis-cloudflare-kv checks if there is a cached result for a query in Redis.
If not found in the cache, it will retrieve data from Cloudflare KV and on the successful result, cache it in Redis for future queries.

Latency Comparison between Cloudflare KV vs Redis

We're calculating latency for Read operation between Cloudflare KV vs Redis. Here Redis Read operation is 6.8X (124.216708ms) times faster than Cloudflare KV (1604.654375s).

package main

import (
    "fmt"
    "log"
    "os"
    "time"

    rediscloudflarekv "github.com/dineshsonachalam/redis-cloudflare-kv"
)

var kvClient = rediscloudflarekv.New(
    os.Getenv("REDIS_URL"),
    os.Getenv("CLOUDFLARE_ACCESS_KEY"),
    os.Getenv("CLOUDFLARE_EMAIL_ADDRESS"),
    os.Getenv("CLOUDFLARE_ACCOUNT_ID"),
)

func measureTime(funcName string) func() {
    start := time.Now()
    return func() {
        fmt.Printf("Time taken by %s function is %v \n", funcName, time.Since(start))
        fmt.Println("==============================================================")
    }
}

func RedisRead(key string) {
    defer measureTime("RedisRead")()
    value, err := kvClient.RedisRead(key)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%v : %v \n", key, string(value))
}

func CloudflareKVRead(key string, namespaceID string) {
    defer measureTime("CloudflareKVRead")()
    value, err := kvClient.CloudflareKVRead(key, namespaceID)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%v : %v \n", key, string(value))
}

func main() {
    key := "opensource.facebook.react-native"
    namespaceID := os.Getenv("TEST_NAMESPACE_ID")
    RedisRead(key)
    CloudflareKVRead(key, namespaceID)
}

// dineshsonachalam@macbook examples % go run latency_check.go
// opensource.facebook.react-native : A framework for building native apps with React.
// Time taken by RedisRead function is 124.216708ms
// ==============================================================
// opensource.facebook.react-native : A framework for building native apps with React.
// Time taken by CloudflareKVRead function is 1.604654375s
// ==============================================================
// dineshsonachalam@macbook examples %

Application Architecture

Installation

You need a working Go environment.

go get github.com/dineshsonachalam/redis-cloudflare-kv

Quickstart

package main

import (
    "fmt"
    "log"
    "os"

    rediscloudflarekv "github.com/dineshsonachalam/redis-cloudflare-kv"
)

func main() {
    // Construct a new KV Client object
    kvClient := rediscloudflarekv.New(
        // REDIS_URL -> TCP Connection (Without SSL):  redis://<user>:<password>@<host>:<port>/<db_number>
        //              TCP Connection (With SSL):     rediss://<user>:<password>@<host>:<port>/<db_number>
        //              UNIX Connection: unix://<user>:<password>@</path/to/redis.sock>?db=<db_number>
        os.Getenv("REDIS_URL"),
        os.Getenv("CLOUDFLARE_ACCESS_KEY"),
        os.Getenv("CLOUDFLARE_EMAIL_ADDRESS"),
        os.Getenv("CLOUDFLARE_ACCOUNT_ID"),
    )
    namespaceID := os.Getenv("TEST_NAMESPACE_ID")
    key1 := "opensource.facebook.react"
    value1 := "A declarative, efficient, and flexible JavaScript library for building user interfaces."
    key2 := "opensource.facebook.react-native"
    value2 := "A framework for building native apps with React."
    key3 := "opensource.facebook.flow"
    value3 := "Adds static typing to JavaScript to improve developer productivity and code quality."
    key4 := "opensource.facebook.docusaurus"
    value4 := "Easy to maintain open source documentation websites."

    // Write writes a value identified by a key in Redis and Cloudflare KV
    status, err := kvClient.Write(key1, []byte(value1), namespaceID)
    if !status && err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Write operation is successful for key: %v\n", key1)

    // Read returns the value associated with the given key
    // If the key is not available in the Redis server,
    // it searches for the key in Cloudflare KV and if the key is available, it writes the key/value in the Redis.
    // Then it returns the value associated with the given key.
    value, err := kvClient.Read(key1, namespaceID)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Read operation is successful. Key: %v, Value: %v\n", key1, string(value))

    // Delete deletes a key/value in Redis and Cloudflare KV
    status, err = kvClient.Delete(key1, namespaceID)
    if !status && err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Delete operation is successful for key: %v\n", key1)

    kvClient.Write(key2, []byte(value2), namespaceID)
    kvClient.Write(key3, []byte(value3), namespaceID)
    kvClient.Write(key4, []byte(value4), namespaceID)

    // ListKeysByPrefix returns keys that matches the prefix
    // If there are no key's that matches the prefix in the Redis
    // We search for the prefix pattern in Cloudflare KV, if there are keys
    // that matches the prefix, we return the keys.
    keys, err := kvClient.ListKeysByPrefix("opensource.facebook", namespaceID)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("ListKeysByPrefix operation is successful. Keys: %v\n", keys)
}

// Output:
// dineshsonachalam@macbook examples % go run main.go
// Write operation is successful for key: opensource.facebook.react
// Read operation is successful. Key: opensource.facebook.react, Value: A declarative, efficient, and flexible JavaScript library for building user interfaces.
// Delete operation is successful for key: opensource.facebook.react
// ListKeysByPrefix operation is successful. Keys: [opensource.facebook.docusaurus opensource.facebook.react-native opensource.facebook.flow]

License

MIT © dineshsonachalam

A simple Python wrapper to AWS Dynamodb

Dinesh — Sat, 24 Jul 2021 16:07:24 +0000

I created a minimalistic python wrapper to AWS Dynamodb.

Github repo: https://github.com/dineshsonachalam/lucid-dynamodb

Installation
Example
- Connect to DynamodDB
- Create a new table
- Get all table names
- Create a New Item
- Read an Item
- Read items by filter
- Update existing attribute in an item
- Add a new attribute in an item
- Add an attribute to the list
- Add an attribute to the string set
- Increase an existing attribute value
- Delete an attribute from an item
- Delete an attribute from the string set
- Delete an item
- Delete a table
Running tests
Github Workflow Artifacts
License

Installation

pip install LucidDynamodb

Note: Prerequisite for Python3 development

Example

Connect to DynamoDB

You can connect to DynamoDB by following any of these two ways.

Using AWS config

from LucidDynamodb import DynamoDb
db = DynamoDb()

"""
$ pip install awscli  #can add user flag
$ aws configure
AWS Access Key ID [****************ABCD]:[enter your key here]
AWS Secret Access Key [****************xyz]:[enter your secret key here]
Default region name [us-west-2]:[enter your region here]
Default output format [None]:
"""

Using AWS secret key

from LucidDynamodb import DynamoDb
import os
AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")
db = DynamoDb(region_name="us-east-1",
              aws_access_key_id=AWS_ACCESS_KEY_ID,
              aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

Create a new table

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    TableAlreadyExists
)
import logging
logging.basicConfig(level=logging.INFO)

table_schema = {
    "TableName": "dev_jobs",
    "KeySchema": [{
            "AttributeName": "company_name",
            "KeyType": "HASH"
        },
        {
            "AttributeName": "role_id",
            "KeyType": "RANGE"
        }
    ],
    "AttributeDefinitions": [{
            "AttributeName": "company_name",
            "AttributeType": "S"
        },
        {
            "AttributeName": "role_id",
            "AttributeType": "S"
        }
    ],
    "GlobalSecondaryIndexes": [],
    "ProvisionedThroughput": {
        "ReadCapacityUnits": 1,
        "WriteCapacityUnits": 1
    }
}

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.create_table(
            table_name=table_schema.get("TableName"),
            key_schema=table_schema.get("KeySchema"),
            attribute_definitions=table_schema.get("AttributeDefinitions"),
            global_secondary_indexes=table_schema.get("GlobalSecondaryIndexes"),
            provisioned_throughput=table_schema.get("ProvisionedThroughput")
        )
        logging.info(f"{table_schema.get('TableName')} table created successfully")
    except TableAlreadyExists as e:
        logging.error(f"{table_schema.get('TableName')} table creation failed - {e}")

"""
dineshsonachalam@macbook examples % python 1-create-a-new-table.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:dev_jobs table created successfully
"""

Get all table names

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        table_names = db.read_all_table_names()
        logging.info(f"Table names: {table_names}")
    except UnexpectedError as e:
        logging.error(f"Read all table names failed - {e}")

"""
dineshsonachalam@macbook examples % python 2-get-all-table-names.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Table names: ['CertMagic', 'dev_jobs', 'dev_test', 'kp-config-v1', 'test-1']
"""

Create a new item

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.create_item(
            table_name="dev_jobs",
            item={
                "company_name": "Google",
                "role_id": "111",
                "role": "Software Engineer 1",
                "salary": "$1,50,531",
                "locations": ["Mountain View, California", "Austin, Texas", "Chicago, IL"],
                "yearly_hike_percent": 8,
                "benefits": set(["Internet, Medical, Edu reimbursements",
                                "Health insurance",
                                "Travel reimbursements"
                                ]),
                "overall_review":{
                    "overall_rating" : "4/5",
                    "compensation_and_benefits": "3.9/5"
                }
            }
        )
        logging.info("Item created successfully")
    except UnexpectedError as e:
        logging.error(f"Item creation failed - {e}")

"""
dineshsonachalam@macbook examples % python 3-create-a-new-item.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Item created successfully
"""

Read an item

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    ItemNotFound
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        item = db.read_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info(f"Item: {item}")
    except ItemNotFound as e:
        logging.error(f"Item doesn't exist - {e}")

"""
dineshsonachalam@macbook examples % python 4-read-an-item.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Item: {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5'
    },
    'company_name': 'Google',
    'role': 'Software Engineer 1',
    'yearly_hike_percent': Decimal('8'),
    'salary': '$1,50,531',
    'benefits': {
        'Travel reimbursements',
        'Internet, Medical, Edu reimbursements',
        'Health insurance'
    }
}
"""

Read items by filter

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    QueryFilterValidationFailed
)
import logging
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.create_item(
            table_name="dev_jobs",
            item={
                "company_name": "Google",
                "role_id": "112",
                "role": "Software Architect",
                "salary": "$4,80,000",
                "locations": ["Mountain View, California"],
                "yearly_hike_percent": 13,
                "benefits": set(["Internet reimbursements"]),
                "overall_review":{
                    "overall_rating" : "3/5",
                    "compensation_and_benefits": "4.2/5"
                }
            }
        )
        logging.info("Item created successfully")
        items = db.read_items_by_filter(
                        table_name='dev_jobs',
                        key_condition_expression=Key("company_name").eq("Google")
        )
        logging.info(f"Items: {items}")
    except QueryFilterValidationFailed as e:
        logging.error(f"Items doesn't exist - {e}")

"""
dineshsonachalam@macbook examples % python 5-read-items-by-filter.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Item created successfully
INFO:root:Items: [{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5'
    },
    'company_name': 'Google',
    'role': 'Software Engineer 1',
    'yearly_hike_percent': Decimal('8'),
    'salary': '$1,50,531',
    'benefits': {
        'Internet, Medical, Edu reimbursements',
        'Travel reimbursements',
        'Health insurance'
    }
}, {
    'locations': ['Mountain View, California'],
    'role_id': '112',
    'overall_review': {
        'compensation_and_benefits': '4.2/5',
        'overall_rating': '3/5'
    },
    'company_name': 'Google',
    'role': 'Software Architect',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$4,80,000',
    'benefits': {
        'Internet reimbursements'
    }
}]
"""

Update existing attribute in an item

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.update_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            },
            attributes_to_update={
                'role': 'Staff Software Engineer 2'
            }
        )
        logging.info("Update is successful")
        item = db.read_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info(f"Item: {item}")
    except UnexpectedError as e:
        logging.error(f"Update failed - {e}")

"""
dineshsonachalam@macbook examples % python 6-update-existing-attribute-in-an-item.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Update is successful
INFO:root:Item: {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5'
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('8'),
    'salary': '$1,50,531',
    'benefits': {
        'Health insurance',
        'Internet, Medical, Edu reimbursements',
        'Travel reimbursements'
    }
}
"""

Add a new attribute in an item

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.update_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            },
            attributes_to_update={
                'overall_review.yearly_bonus_percent': 12
            }
        )
        logging.info("Update is successful")
        item = db.read_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info(f"Item: {item}")
    except UnexpectedError as e:
        logging.error(f"Update failed - {e}")

"""
dineshsonachalam@macbook examples % python 7-add-a-new-attribute-in-an-item.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Update is successful
INFO:root:Item: {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('8'),
    'salary': '$1,50,531',
    'benefits': {
        'Travel reimbursements',
        'Internet, Medical, Edu reimbursements',
        'Health insurance'
    }
}
"""

Add an attribute to the list

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.update_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            },
            attributes_to_update={
                'locations': "Detroit, Michigan"
            },
            operation="ADD_ATTRIBUTE_TO_LIST"
        )
        logging.info("Update is successful")
        item = db.read_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info(f"Item: {item}")
    except UnexpectedError as e:
        logging.error(f"Update failed - {e}")

"""
dineshsonachalam@macbook examples % python 8-add-an-attribute-to-the-list.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Update is successful
INFO:root:Item: {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('8'),
    'salary': '$1,50,531',
    'benefits': {
        'Health insurance',
        'Travel reimbursements',
        'Internet, Medical, Edu reimbursements'
    }
}
"""

Add an attribute to the string set

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.update_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            },
            attributes_to_update={
                'benefits': "Free Food"
            },
            operation="ADD_ATTRIBUTE_TO_STRING_SET"
        )
        logging.info("Update is successful")
        item = db.read_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info(f"Item: {item}")
    except UnexpectedError as e:
        logging.error(f"Update failed - {e}")

"""
dineshsonachalam@macbook examples % python 9-add-an-attribute-to-the-string-set.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Update is successful
INFO:root:Item: {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('8'),
    'salary': '$1,50,531',
    'benefits': {
        'Travel reimbursements',
        'Free Food',
        'Health insurance',
        'Internet, Medical, Edu reimbursements'
    }
}
"""

Increase an existing attribute value

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.increase_attribute_value(
            table_name='dev_jobs',
            key={
                "company_name": "Google",
                "role_id": "111"
            },
            attribute_name="yearly_hike_percent",
            increment_value=5
        )
        logging.info("Attribute value increment completed")
        item = db.read_item(
            table_name='dev_jobs',
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info(f"Item: {item}")
    except UnexpectedError as e:
        logging.error(f"Attribute value increment failed - {e}")

"""
dineshsonachalam@macbook examples % python 10-increase-an-existing-attribute-value.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Attribute value increment completed
INFO:root:Item: {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$1,50,531',
    'benefits': {
        'Internet, Medical, Edu reimbursements',
        'Free Food',
        'Health insurance',
        'Travel reimbursements'
    }
}
"""

Delete an attribute from an item

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.delete_attribute(
            table_name="dev_jobs",
            key={"company_name": "Google", "role_id": "111"},
            attribute_name="yearly_hike_percent")
        logging.info("The attribute is deleted successfully")
        item = db.read_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info(f"Item: {item}")
    except UnexpectedError as e:
         logging.error(f"The attribute delete operation failed - {e}")

"""
dineshsonachalam@macbook examples % python 11-delete-an-attribute-from-an-item.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:The attribute is deleted successfully
INFO:root:Item: {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'salary': '$1,50,531',
    'benefits': {
        'Travel reimbursements',
        'Free Food',
        'Health insurance',
        'Internet, Medical, Edu reimbursements'
    }
}
"""

Delete an attribute from the string set

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.update_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            },
            attributes_to_update={
                'benefits': "Free Food"
            },
            operation="DELETE_ATTRIBUTE_FROM_STRING_SET"
        )
        logging.info("Update is successful")
        item = db.read_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info(f"Item: {item}")
    except UnexpectedError as e:
        logging.error(f"Update failed - {e}")

"""
dineshsonachalam@macbook examples % python 12-delete-an-attribute-from-the-string-set.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Update is successful
INFO:root:Item: {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': '111',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'salary': '$1,50,531',
    'benefits': {
        'Internet, Medical, Edu reimbursements',
        'Health insurance',
        'Travel reimbursements'
    }
}
"""

Delete an item

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    UnexpectedError
)
from boto3.dynamodb.conditions import Key
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.delete_item(
            table_name="dev_jobs",
            key={
                "company_name": "Google",
                "role_id": "111"
            }
        )
        logging.info("Item deleted successfully")
        items = db.read_items_by_filter(
                        table_name='dev_jobs',
                        key_condition_expression=Key("company_name").eq("Google")
        )
        logging.info(f"Items: {items}") 
    except UnexpectedError as e:
        logging.warning(f"Item delete operation failed - {e}")

"""
dineshsonachalam@macbook examples % python 13-delete-an-item.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Item deleted successfully
INFO:root:Items: [{
    'locations': ['Mountain View, California'],
    'role_id': '112',
    'overall_review': {
        'compensation_and_benefits': '4.2/5',
        'overall_rating': '3/5'
    },
    'company_name': 'Google',
    'role': 'Software Architect',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$4,80,000',
    'benefits': {
        'Internet reimbursements'
    }
}]
"""

Delete a table

from LucidDynamodb import DynamoDb
from LucidDynamodb.exceptions import (
    TableNotFound
)
import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    try:
        db = DynamoDb()
        db.delete_table(table_name='dev_jobs')
        logging.info("Table deleted successfully")
        table_names = db.read_all_table_names()
        logging.info(f"Table names: {table_names}")
    except TableNotFound as e:
        logging.error(f"Table delete operation failed {e}")

"""
dineshsonachalam@macbook examples % python 14-delete-a-table.py
INFO:botocore.credentials:Found credentials in environment variables.
INFO:root:Table deleted successfully
INFO:root:Table names: ['CertMagic', 'dev_test', 'kp-config-v1', 'test-1']
"""

Running Tests

To run tests, run the following command

pytest -s

License

MIT © dineshsonachalam

A GitHub Action that automatically generates & updates markdown content (like your README.md) from external or remote files.

Dinesh — Sat, 24 Jul 2021 16:01:43 +0000

Why markdown-autodocs?
Features
Examples
- CODE Block
- JSON to HTML table
- Github Workflow Artifacts table
Example Repo which uses all the markdown-autodocs feature
Local usage without Github action
Usage
- Adding markdown-autodocs in your workflow
- Extended example with all possible options available for this Action
Github Workflow Artifacts
Contributing
Used By
License

Why markdown-autodocs

To make your repo more appealing and useful you need to provide example code snippets in your README.md. Manually copy and pasting each code snippet in their respective places in your README would be inefficient and time-consuming.

This problem can be solved using markdown-autodocs a GitHub Action that automatically generates & updates markdown content (like your README.md) from external or remote files. You need to add markers in your README.md that will tell markdown-autodocs where to insert the code snippet.

Features

Code block: Insert code snippet in your markdown file from external or remote files.
JSON to HTML table: Insert HTML Table in your markdown file by converting JSON file contents to HTML table.
Github Workflow Artifacts table: Insert the Github workflow artifacts table in your markdown file by getting the latest artifacts for a workflow run.

Examples

CODE Block

Get code from an external file or URL and insert it in your markdown.

Options:

src: The relative path to the code to pull in, or the URL where the raw code lives

JSON to HTML table

Get JSON contents from an external file and convert it into an HTML table and insert's it in your markdown.

Options:

src: The relative path to the JSON file to pull in.

Github Workflow Artifacts table

Get the list of the latest artifacts generated from a workflow run. Generates a workflow artifacts table consists of artifacts download and workflow URL in an HTML table and inserts it in your markdown file.

Example Repo which uses all the markdown-autodocs feature

Local usage without Github action

Install markdown-autodocs CLI:

npm i -g markdown-autodocs

markdown-autodocs CLI usage:

dineshsonachalam@macbook ~ % markdown-autodocs --help
Usage: markdown-autodocs [options]

Options:
  -o, --outputFilePath <outputFilePaths...>  Output file paths
  -c, --category <categories...>             code-block, json-to-html-table, workflow-artifact-table
  -r, --repo <type>                          Repo name
  -b, --branch <type>                        Branch name
  -a, --accessToken <type>                   Github Access token
  -h, --help                                 display help for command

Code block

markdown-autodocs -c code-block -o ./README.md

JSON to HTML table

markdown-autodocs -c json-to-html-table -o ./README.md

Github workflow artifacts table

markdown-autodocs -c workflow-artifact-table -o ./README.md -r $REPO -b $BRANCH -a $ACCESSTOKEN

Usage

Adding markdown-autodocs in your workflow

Add the following step at the end of your job, after other steps that might add or change files.

uses: dineshsonachalam/markdown-autodocs@v1.0.3

Extended example with all possible options available for this Action

- name: Markdown autodocs
- uses: dineshsonachalam/markdown-autodocs@v1.0.3
  with:
    # Optional, defaults to author of the commit that triggered the run
    commit_author: Author <actions@github.com>

    # Optional, defaults to "actions@github.com"
    commit_user_email: actions@github.com

    # Optional, but recommended
    # Defaults to "Apply automatic changes"
    commit_message: Apply automatic changes

    # Optional branch name where commit should be pushed to.
    # Defaults to the current branch.
    branch: feature-123

    # Optional output file paths, defaults to '[./README.md]'.
    output_file_paths: '[./README.md]'

    # Categories to automatically sync or transform its contents in the markdown files.
    # Defaults to '[code-block,json-to-html-table,workflow-artifact-table]'
    categories: '[code-block,json-to-html-table,workflow-artifact-table]'

🚀 Used By

License

MIT © dineshsonachalam

Markdown automation made simple using markdown-autodocs github action

Dinesh — Mon, 28 Jun 2021 01:29:05 +0000

Hi everyone!, I just released markdown-autodocs GitHub action which will help you to auto-document your markdown files. Please give a star for this repo if you find it useful.

Repo: https://github.com/dineshsonachalam/markdown-autodocs

Why markdown-autodocs?
Features
Examples
- CODE Block
- JSON to HTML table
- Github Workflow Artifacts table
- Example Repo which uses all the markdown-autodocs feature
Usage
- Adding markdown-autodocs in your workflow
- Extended example with all possible options available for this Action
Github Workflow Artifacts
Contributing
License

Why markdown-autodocs?

Features

Code block: Insert code snippet in your markdown file from external or remote files.
JSON to HTML table: Insert HTML Table in your markdown file by converting JSON file contents to HTML table.
Github Workflow Artifacts table: Insert the Github workflow artifacts table in your markdown file by getting the latest artifacts for a workflow run.

Examples

CODE Block

Get code from an external file or URL and insert it in your markdown.

Options:

src: The relative path to the code to pull in, or the URL where the raw code lives

Get code from external file

Get code from remote file

JSON to HTML table

Get JSON contents from an external file and convert it into an HTML table and insert's it in your markdown.

Options:

src: The relative path to the JSON file to pull in.

Github Workflow Artifacts table

Example Repo which uses all the markdown-autodocs feature

Usage

Adding markdown-autodocs in your workflow

Add the following step at the end of your job, after other steps that might add or change files.

uses: dineshsonachalam/markdown-autodocs@v1.0.2

Extended example with all possible options available for this Action

- name: Markdown autodocs
- uses: dineshsonachalam/markdown-autodocs@v1.0.2
  with:
    # Optional, defaults to author of the commit that triggered the run
    commit_author: Author <actions@github.com>

    # Optional, defaults to "actions@github.com"
    commit_user_email: actions@github.com

    # Optional, but recommended
    # Defaults to "Apply automatic changes"
    commit_message: Apply automatic changes

    # Optional branch name where commit should be pushed to.
    # Defaults to the current branch.
    branch: feature-123

    # Optional output file paths, defaults to '[./README.md]'.
    output_file_paths: '[./README.md]'

    # Categories to automatically sync or transform its contents in the markdown files.
    # Defaults to '[code-block,json-to-html-table,workflow-artifact-table]'
    categories: '[code-block,json-to-html-table,workflow-artifact-table]'

Github Workflow Artifacts

Artifact	Workflow
Jest-integration-test-report	integration-tests
module-dependencies-license-report	integration-tests
size-of-dependencies	integration-tests
vulnerabilities-audit-report	integration-tests

Contributing

License

A minimalistic Python wrapper to AWS Dynamodb

Dinesh — Tue, 01 Jun 2021 06:17:23 +0000

I wrote a simple wrapper to AWS DynamoDB. It covers all the DynamoDB operations. Please give a star for this repository, if you find it useful.
Reference: https://github.com/dineshsonachalam/Lucid-Dynamodb

Installation

pip install LucidDynamodb

Note: Prerequisite for Python3 development

Example
- Create a new table
- Get all table names
- Create a New Item
- Read an Item
- Increase an existing attribute value
- Update existing attribute in an item
- Add a new attribute in an item
- Add an attribute to the list
- Add an attribute to the string set
- Delete an attribute from the string set
- Delete an attribute from an item
- Read items by filter
- Delete a table

Example

1. Create a new table

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

table_schema = {
    "TableName": "dev_jobs",
    "KeySchema": [
        {
            "AttributeName": "company_name",
            "KeyType": "HASH"
        },
        {
            "AttributeName": "role_id",
            "KeyType": "RANGE"
        }
    ],
    "AttributeDefinitions": [
        {
            "AttributeName": "company_name",
            "AttributeType": "S"
        },
        {
            "AttributeName": "role_id",
            "AttributeType": "S"
        }
     ],
    "GlobalSecondaryIndexes": [],
    "ProvisionedThroughput": {
        "ReadCapacityUnits": 1,
        "WriteCapacityUnits": 1
    }
}


if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    table_creation_status = db.create_table(
                                    TableName=table_schema.get("TableName"),
                                    KeySchema=table_schema.get("KeySchema"),
                                    AttributeDefinitions=table_schema.get("AttributeDefinitions"),
                                    GlobalSecondaryIndexes=table_schema.get("GlobalSecondaryIndexes"),
                                    ProvisionedThroughput=table_schema.get("ProvisionedThroughput")
    )
    if(table_creation_status == True):
        logging.info("{} table created successfully".format(table_schema.get("TableName")))
    else:
        logging.error("{} table creation failed".format(table_schema.get("TableName")))

Output:

INFO:root:dev_jobs table created successfully

2. Get all table names

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)
    table_names = db.read_all_table_names()
    logging.info("Table names: {}".format(table_names))

Output:

INFO:root:Table names: ['dev_jobs', 'user']

3. Create a New Item

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    item_creation_status = db.create_item(
        TableName="dev_jobs", 
        Item={
            "company_name": "Google",
            "role_id": str(uuid.uuid4()),
            "role": "Software Engineer 1",
            "salary": "$1,50,531",
            "locations": ["Mountain View, California", "Austin, Texas", "Chicago, IL"],
            "yearly_hike_percent": 8,
            "benefits": set(["Internet, Medical, Edu reimbursements", 
                             "Health insurance",
                             "Travel reimbursements"
                             ]),
            "overall_review":{
                "overall_rating" : "4/5",
                "compensation_and_benefits": "3.9/5"
            }
        }
    )
    if(item_creation_status == True):
        logging.info("Item created successfully")
    else:
        logging.warning("Item creation failed")

Output:

INFO:root:Item created successfully

4. Read an Item

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)
    item = db.read_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        })
    if(item != None):
        logging.info("Item: {}".format(item))
    else:
        logging.warning("Item doesn't exist")

Output:

INFO:root:Item: 
{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5'
    },
    'company_name': 'Google',
    'role': 'Software Engineer 1',
    'yearly_hike_percent': Decimal('8'),
    'salary': '$1,50,531',
    'benefits': {
        'Health insurance',
        'Travel reimbursements',
        'Internet, Medical, Edu reimbursements'
    }
}

5. Increase an existing attribute value

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    increase_attribute_status = db.increase_attribute_value(
        TableName='dev_jobs',
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        },
        AttributeName="yearly_hike_percent",
        IncrementValue=5
    )

    if(increase_attribute_status==True):
        logging.info("Attribute value increment completed")
    else:
        logging.warning("Attribute value increment failed")

    item = db.read_item(
        TableName='dev_jobs', 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        })
    if(item != None):
        logging.info("Item: {}".format(item))
    else:
        logging.warning("Item doesn't exist")

Output:

INFO:root: Attribute value increment completed
INFO:root: Item: 
{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5'
    },
    'company_name': 'Google',
    'role': 'Software Engineer 1',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$1,50,531',
    'benefits': {
        'Health insurance',
        'Travel reimbursements',
        'Internet, Medical, Edu reimbursements'
    }
}

6. Update existing attribute in an item

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    item_update_status = db.update_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        },
        AttributesToUpdate={
            'role': 'Staff Software Engineer 2'
        }
    )
    if(item_update_status == True):
        logging.info("Update is successful")
    else:
        logging.warning("Update failed")

    item = db.read_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        })
    if(item != None):
        logging.info("Item: {}".format(item))
    else:
        logging.warning("Item doesn't exist")

Output:

INFO:root:Update is successful
INFO:root:Item: 
{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5'
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$1,50,531',
    'benefits': {
        'Internet, Medical, Edu reimbursements',
        'Travel reimbursements',
        'Health insurance'
    }
}

7. Add a new attribute in an item

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    item_update_status = db.update_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        },
        AttributesToUpdate={
            'overall_review.yearly_bonus_percent': 12
        }
    )
    if(item_update_status == True):
        logging.info("Update is successful")
    else:
        logging.warning("Update failed")

    item = db.read_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        })
    if(item != None):
        logging.info("Item: {}".format(item))
    else:
        logging.warning("Item doesn't exist")

Output:

INFO:root:Update is successful
INFO:root:Item: 
{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$1,50,531',
    'benefits': {
        'Internet, Medical, Edu reimbursements',
        'Travel reimbursements',
        'Health insurance'
    }
}

8. Add an attribute to the list

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    item_update_status = db.update_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        },
        AttributesToUpdate={
            'locations': "Detroit, Michigan"
        },
        Operation="ADD_ATTRIBUTE_TO_LIST"
    )
    if(item_update_status == True):
        logging.info("Update is successful")
    else:
        logging.warning("Update failed")

    item = db.read_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        })
    if(item != None):
        logging.info("Item: {}".format(item))
    else:
        logging.warning("Item doesn't exist")

Output:

INFO:root:Update is successful
INFO:root:Item: 
{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$1,50,531',
    'benefits': {
        'Health insurance',
        'Internet, Medical, Edu reimbursements',
        'Travel reimbursements'
    }
}

9. Add an attribute to the string set

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    item_update_status = db.update_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        },
        AttributesToUpdate={
            'benefits': "Free Food"
        },
        Operation="ADD_ATTRIBUTE_TO_STRING_SET"
    )
    if(item_update_status == True):
        logging.info("Update is successful")
    else:
        logging.warning("Update failed")

    item = db.read_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        })
    if(item != None):
        logging.info("Item: {}".format(item))
    else:
        logging.warning("Item doesn't exist")

Output:

INFO:root:Update is successful
INFO:root:Item: 
{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$1,50,531',
    'benefits': {
        'Internet, Medical, Edu reimbursements',
        'Health insurance',
        'Free Food',
        'Travel reimbursements'
    }
}

10. Delete an attribute from the string set

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    item_update_status = db.update_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        },
        AttributesToUpdate={
            'benefits': "Free Food"
        },
        Operation="DELETE_ATTRIBUTE_FROM_STRING_SET"
    )
    if(item_update_status == True):
        logging.info("Update is successful")
    else:
        logging.warning("Update failed")

    item = db.read_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        })
    if(item != None):
        logging.info("Item: {}".format(item))
    else:
        logging.warning("Item doesn't exist")

Output:

INFO:root:Update is successful
INFO:root:Item: 
{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$1,50,531',
    'benefits': {
        'Health insurance',
        'Internet, Medical, Edu reimbursements',
        'Travel reimbursements'
    }
}

11. Delete an attribute from an item

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    attribute_delete_status = db.delete_attribute(
        TableName="dev_jobs", 
        Key={
              "company_name": "Google",
              "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        }, 
        AttributeName="yearly_hike_percent")

    if(attribute_delete_status == True):
        logging.info("The attribute is deleted successfully")
    else:
        logging.warning("The attribute delete operation failed")

    item = db.read_item(
        TableName="dev_jobs", 
        Key={
            "company_name": "Google",
            "role_id": "e85f79a7-0857-4086-afbd-da13ec76b442"
        })
    if(item != None):
        logging.info("Item: {}".format(item))
    else:
        logging.warning("Item doesn't exist")

Output:

INFO:root:The attribute is deleted successfully
INFO:root:Item: 
{
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'salary': '$1,50,531',
    'benefits': {
        'Internet, Medical, Edu reimbursements',
        'Travel reimbursements',
        'Health insurance'
    }
}

12. Read items by filter

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    item_creation_status = db.create_item(
        TableName="dev_jobs", 
        Item={
            "company_name": "Google",
            "role_id": str(uuid.uuid4()),
            "role": "Software Architect",
            "salary": "$4,80,000",
            "locations": ["Mountain View, California"],
            "yearly_hike_percent": 13,
            "benefits": set(["Internet reimbursements"]),
            "overall_review":{
                "overall_rating" : "3/5",
                "compensation_and_benefits": "4.2/5"
            }
        }
    )
    if(item_creation_status == True):
        logging.info("Item created successfully")
    else:
        logging.warning("Item creation failed")

    items = db.read_items_by_filter(
                    TableName='dev_jobs', 
                    KeyConditionExpression=Key("company_name").eq("Google") 
    )

    if(len(items)>0):
        logging.info("Items: {}".format(items))
    else:
        logging.warning("Items doesn't exist")

Output:

INFO:root: Item created successfully
INFO:root:Items: 
[{
    'locations': ['Mountain View, California'],
    'role_id': 'b6065b19-4333-43a4-abf7-dedca2880669',
    'overall_review': {
        'compensation_and_benefits': '4.2/5',
        'overall_rating': '3/5'
    },
    'company_name': 'Google',
    'role': 'Software Architect',
    'yearly_hike_percent': Decimal('13'),
    'salary': '$4,80,000',
    'benefits': {
        'Internet reimbursements'
    }
}, {
    'locations': ['Mountain View, California', 'Austin, Texas', 'Chicago, IL', 'Detroit, Michigan'],
    'role_id': 'e85f79a7-0857-4086-afbd-da13ec76b442',
    'overall_review': {
        'compensation_and_benefits': '3.9/5',
        'overall_rating': '4/5',
        'yearly_bonus_percent': Decimal('12')
    },
    'company_name': 'Google',
    'role': 'Staff Software Engineer 2',
    'salary': '$1,50,531',
    'benefits': {
        'Health insurance',
        'Internet, Medical, Edu reimbursements',
        'Travel reimbursements'
    }
}]

13. Delete a table

from LucidDynamodb.Operations import DynamoDb
import os
import logging
import uuid
from boto3.dynamodb.conditions import Key
logging.basicConfig(level=logging.INFO)

AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")

if __name__ == "__main__":
    db = DynamoDb(region_name="us-east-1", 
                aws_access_key_id=AWS_ACCESS_KEY_ID, 
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY)

    delete_table_status = db.delete_table(TableName='dev_jobs')
    if(delete_table_status == True):
        logging.info("Table deleted successfully")
    else:
        logging.warning("Table delete operation failed")

    table_names = db.read_all_table_names()
    logging.info("Table names: {}".format(table_names))

Output:

INFO:root: Table deleted successfully
INFO:root:Table names: ['user']

License

This project is licensed under the terms of the MIT license.

Forem: Dinesh

A real-time tech course finder, created using Elasticsearch, Python, React+Redux, Docker, and Kubernetes.

Demo

What's this project all about?

Application Architecture

1. Understanding all significant components in ElasticSearch and it's Auto completion feature.

2. Building an API service that interacts with ElasticSearch to be used by the UI.

ElasticSearch Autocomplete

Query Search

3. Testing API using Pytest

4. Building UI using React and Redux.

5. Testing UI using Cypress.

License

Redis caching layer for Cloudflare KV in Golang

Why redis-cloudflare-kv?

Latency Comparison between Cloudflare KV vs Redis

Application Architecture

Installation

Quickstart

License

A simple Python wrapper to AWS Dynamodb

Table of contents

Installation

Example

Connect to DynamoDB

Create a new table

Get all table names

Create a new item

Read an item

Read items by filter

Update existing attribute in an item

Add a new attribute in an item

Add an attribute to the list

Add an attribute to the string set

Increase an existing attribute value

Delete an attribute from an item

Delete an attribute from the string set

Delete an item

Delete a table

Running Tests

License

A GitHub Action that automatically generates & updates markdown content (like your README.md) from external or remote files.

Table of contents

Why markdown-autodocs

Features

Examples

CODE Block

JSON to HTML table

Github Workflow Artifacts table

Example Repo which uses all the markdown-autodocs feature

Local usage without Github action

Usage

Adding markdown-autodocs in your workflow

Extended example with all possible options available for this Action

🚀 Used By

License

Markdown automation made simple using markdown-autodocs github action

Table of contents

Why markdown-autodocs?

Features

Examples

CODE Block

JSON to HTML table

Github Workflow Artifacts table

Example Repo which uses all the markdown-autodocs feature

Usage

Adding markdown-autodocs in your workflow

Extended example with all possible options available for this Action

Github Workflow Artifacts

Contributing

License

A minimalistic Python wrapper to AWS Dynamodb

Installation

Table of Contents

Example

1. Create a new table

2. Get all table names

3. Create a New Item

4. Read an Item

5. Increase an existing attribute value