Forem: Yuki Ogawa

Fixing AWS Login Credentials Expiring After 15 Minutes: The awscrt Library Mystery

Yuki Ogawa — Sun, 18 Jan 2026 13:43:26 +0000

Fixing AWS Login Credentials Expiring After 15 Minutes: The awscrt Library Mystery

Introduction

I was having trouble with my Python application using boto3 - even though I authenticated with the aws login command, I kept getting "credentials expired" errors after about 15 minutes.

After investigation, I discovered that an outdated version of the awscrt library was causing automatic credential refresh to fail.

This article shares the root cause and solution. I hope it helps anyone facing the same issue!

What is aws login?

aws login is a relatively new command added in AWS CLI 2.15.0.

Differences from Traditional Authentication Methods

Traditional method (aws sso login):

aws sso login --profile my-profile

Uses AWS IAM Identity Center (formerly AWS SSO)
Requires accessing your organization's SSO portal for authentication

New method (aws login):

aws login

Uses AWS Management Console credentials directly
Same experience as logging into the AWS Console via browser
More intuitive and simpler

Features of aws login

🔐 Refresh token support: Once logged in, credentials should automatically refresh
🚀 Simple: No profile configuration needed
🔄 boto3 integration: Works transparently with Python applications

The Problem

Symptoms

My Python application using boto3 was experiencing the following issue:

import boto3

session = boto3.Session()
sts = session.client('sts')

# Initially works
identity = sts.get_caller_identity()
print(f"Account: {identity['Account']}")  # OK

# 15 minutes later...
identity = sts.get_caller_identity()  # ❌ Error!

Error message:

ExpiredTokenException: The security token included in the request is expired

Expected Behavior

boto3 should automatically use the refresh token to obtain new credentials when the current ones are about to expire.

However, in reality, the refresh was failing, and credentials were expiring after 15 minutes.

Investigation

1. Checking Credentials

First, I verified which credentials boto3 was using:

import boto3

session = boto3.Session()
credentials = session.get_credentials()

print(f"Provider: {credentials.method}")
# Output: login

print(f"Refreshable: {isinstance(credentials, RefreshableCredentials)}")
# Output: True

boto3 was correctly recognizing the login provider and treating it as refreshable credentials.

2. Checking Cache Files

aws login credentials are cached at:

~/.aws/login/cache/<hash>.json

The contents showed that the refresh token was properly stored:

{
  "accessToken": {
    "accessKeyId": "ASIA************",
    "secretAccessKey": "**********************",
    "sessionToken": "**********************",
    "expiresAt": "2026-01-05T07:12:30Z"
  },
  "refreshToken": "**********************",
  "dpopKey": "-----BEGIN EC PRIVATE KEY-----\n..."
}

3. Testing Refresh Behavior

I forced the expiration time to be shorter to see if refresh would execute:

import boto3
import json
from datetime import datetime, timedelta, timezone

# Load cache file
cache_file = "~/.aws/login/cache/<hash>.json"
with open(cache_file, 'r') as f:
    token_data = json.load(f)

# Set expiration to 1 minute from now
new_expiry = datetime.now(timezone.utc) + timedelta(minutes=1)
token_data['accessToken']['expiresAt'] = new_expiry.strftime('%Y-%m-%dT%H:%M:%SZ')

# Update cache file
with open(cache_file, 'w') as f:
    json.dump(token_data, f)

# Create new boto3 session
session = boto3.Session()
sts = session.client('sts')

# STS call (should trigger refresh)
identity = sts.get_caller_identity()

This resulted in the following error:

AttributeError: type object 'EC' has no attribute 'decode_der_signature_to_padded_pair'

This was the root cause!

4. Identifying the Root Cause

Examining boto3's source code revealed that aws login refresh processing requires generating a DPoP (Demonstrating Proof-of-Possession) header.

This header generation uses the EC class from the awscrt library, but the old version (0.28.4) was missing the required method.

# Simplified excerpt from botocore/credentials.py
from awscrt.auth import EC

def _build_dpop_header(private_key, uri):
    # ...
    signature_bytes = EC.decode_der_signature_to_padded_pair(
        signature_der, EC.CurveType.SECP384R1
    )  # ← This method doesn't exist in awscrt 0.28.4!

Solution

Upgrading Libraries

The solution was simple: upgrade the awscrt library:

pip install --upgrade awscrt botocore boto3

Versions After Upgrade

awscrt: 0.28.4 → 0.30.0
botocore: 1.42.19 → 1.42.21
boto3: 1.42.19 → 1.42.21

Adding to requirements.txt

To prevent this issue in the future, I specified minimum versions in requirements.txt:

# AWS credentials refresh support
# awscrt >= 0.30.0 is required for aws login credential auto-refresh
awscrt>=0.30.0
boto3>=1.42.21
botocore>=1.42.21

Verification

Test Script

After upgrading, I verified that refresh was working properly:

# test_credential_refresh.py
import boto3
import json
from datetime import datetime, timedelta, timezone

print("=" * 80)
print("Credential Auto-Refresh Test")
print("=" * 80)

# Cache file path
cache_file = "~/.aws/login/cache/<hash>.json"

# Set expiration to 1 minute from now (force refresh)
with open(cache_file, 'r') as f:
    token_data = json.load(f)

original_access_key = token_data['accessToken']['accessKeyId']
print(f"Original access key: ASIA***************...")

new_expiry = datetime.now(timezone.utc) + timedelta(minutes=1)
token_data['accessToken']['expiresAt'] = new_expiry.strftime('%Y-%m-%dT%H:%M:%SZ')

with open(cache_file, 'w') as f:
    json.dump(token_data, f)

# Create boto3 session
session = boto3.Session()
sts = session.client('sts')

# STS call (triggers refresh)
print("\nExecuting STS call...")
identity = sts.get_caller_identity()
print(f"✓ Success: {identity['Account']}")

# Check cache file after refresh
with open(cache_file, 'r') as f:
    refreshed_token = json.load(f)

refreshed_access_key = refreshed_token['accessToken']['accessKeyId']
print(f"\nRefreshed access key: ASIA***************...")

if refreshed_access_key != original_access_key:
    print("\n✓✓✓ New credentials obtained!")
    print("Refresh is working properly!")
else:
    print("\n✗ Credentials were not updated")

print("=" * 80)

Execution Results

================================================================================
Credential Auto-Refresh Test
================================================================================
Original access key: ASIA***************...

Executing STS call...
✓ Success: **********

Refreshed access key: ASIA***************...

✓✓✓ New credentials obtained!
Refresh is working properly!
================================================================================

Perfect! New access keys were obtained, confirming that refresh is working properly.

boto3 Refresh Timing

boto3 attempts to refresh credentials at the following times:

Timing	Description	Behavior on Failure
Advisory Refresh	15 minutes before expiration	Continue using existing credentials
Mandatory Refresh	10 minutes before expiration	Return error
aws login specific	5 minutes before expiration	Return error

In other words, automatic refresh occurs 5 minutes before expiration, resolving the 15-minute expiration issue.

Summary

Root Cause

The reason aws login credentials were expiring after 15 minutes was an outdated awscrt library version.

awscrt 0.28.4 was missing the method needed for DPoP header generation
This caused errors during refresh
boto3 failed to auto-update, resulting in expired credentials

Solution

pip install --upgrade awscrt botocore boto3

Just this simple command made credential auto-refresh work properly!

Key Points

✅ awscrt >= 0.30.0 is required
✅ Latest boto3/botocore versions recommended
✅ Credentials auto-refresh 5 minutes before expiration
✅ No re-login needed as long as refresh token is valid

References

Conclusion

Since aws login is a relatively new feature, there's still limited information available. When I asked Kiro about it, I was repeatedly directed to aws sso login and modifications to credentials and config files...

I hope this article helps anyone encountering the same issue!

Image Generation Chat App using Amazon Nova Canvas

Yuki Ogawa — Sun, 11 May 2025 13:56:38 +0000

Introduction

The Amazon Nova series announced at AWS re:Invent 2024 includes Nova Canvas, which can generate images. Previously, to generate images from Amazon Bedrock, the options were limited to the Amazon Titan series or the Stable Diffusion series, but now Nova Canvas has been added as a new choice.

In this article, we will explore what Nova Canvas can do and how to create an image generation chatbot.

Features of Nova Canvas

Nova Canvas is a generative AI model that creates new images from text or image prompts. The features of Nova Canvas are as follows:

Feature	Description
Provision of Reference Images	Can provide reference images useful for generating images or videos
Determination of Color Palette	Determines the color scheme or "color palette" of an image using text input
Image Editing	Allows replacing objects or backgrounds in input images using text prompts
Background Removal	Easily removes backgrounds, leaving the subject of the image unchanged
Safety, Responsible AI, Compensation	Includes traceability, content moderation, and watermarks for compensation

https://aws.amazon.com/ai/generative-ai/nova/creative/

Nova Canvas vs Titan

Below is a comparison between the Nova series and the Titan series.
The Nova series has been enhanced to handle longer content and more complex documents overall.

Feature	Titan Model	Nova Model
Optimal Use Case	General text generation, image tasks, embeddings	Long content and complex document processing
Use Case Scenarios	When image generation or embeddings are needed	When processing very long documents
Strengths	Versatility in text, image, and embeddings	Larger context window (up to 300K tokens)
Cost	Relatively low cost	High cost
Application Integration	Broad integration possibilities	Often optimized for specific use cases
Low Latency for Standard Tasks	Yes	No
Optimized for Production Workloads	Yes	No
Standard Context Length	Yes	No
Fast Response Time Required	Yes	No

On-Demand Pricing for Nova Canvas

Below are the usage fees for Nova Canvas and Titan Image Generator in the us-east-1 (Northern Virginia) region.
Depending on the image size, Nova Canvas costs several times more than Titan, so it is necessary to use them according to the use case.

Titan Image Generator also has a v1, but the cost in the above region was the same at the time of the survey.

Model	Image Resolution	Price per Image Generated at Standard Quality	Price per Image Generated at Premium Quality
Amazon Nova Canvas	Up to 1024 x 1024	USD 0.04	USD 0.06
Amazon Nova Canvas	Up to 2048 x 2048	USD 0.06	USD 0.08
Amazon Titan Image Generator v2	Smaller than 512 x 512	USD 0.008	USD 0.01
Amazon Titan Image Generator v2	Larger than 1024 x 1024	USD 0.01	USD 0.012

https://aws.amazon.com/bedrock/pricing/

Nova Canvas Chat App

Now, let's create a chatbot app using Nova Canvas!
Here is the technology stack we will use.

Item	Content	Remarks
Client Side	Streamlit	Web framework implemented in Python
Server Side	Cloud9	Integrated IDE environment on AWS, but new usage has been stopped. Alternatively, use VSCode Server or Amazon SageMaker Studio code editor.
Language	Python	Use version 3.9 or higher

Additionally, this time we will create a simple chatbot that can create icons with instructions in any languages.
The settings for the images to be created are just setting system prompts and negative prompts, so change the prompts according to the image you want to create.

1. Set Up the Development Environment

First, let's set up the development environment.
Install Python version


 or higher and create a development directory.

Install the following packages with

 ```pip install -r requirements.txt```

.



```requirements.txt
boto3==1.38.6
streamlit==1.45.0

The versions seem to be fine even if they are the latest, but please check before proceeding.

2. Create the Chat App

System Architecture

Here is the architecture of the chat app we will create. Initially, I thought it would be fine to just call Nova Canvas from Streamlit via boto3, but to incorporate the way of giving prompts for image generation, I added a process to extract keywords from the instruction sentences in the chat.

Tips for Image Generation via Gen-AI

Give Instructions with Words Instead of Sentences

Since generative AI understands prompt sentences by dividing them into a certain context length, it is better to give instructions with important words separated by commas.
Of course, Nova Canvas has the advantage of understanding longer contexts compared to Titan, but by using a description method that is easy for the generative AI model to understand, you can create images as you envision.

Example: Surprised child, colorful playground, anime style

Bring Important Words to the Front

It is necessary to bring important expressions to the front among the things you want to draw. Since the main theme this time is simple icon generation, the content defined as system prompts is brought to the front of the prompt sentence.

Utilize Negative Prompts

Negative prompts are important in image generation. Especially in normal prompt sentences, it is important to describe things you do not want to include in negative prompts instead of using negative expressions like not or no.
If negative expressions are included in the prompt, the generative AI model may generate an image that includes those words.

In this script, since we want to generate simple icons, we have added 3D, color, photo to the default negative prompts.
Also, since instructing icon generation may create multiple icon sets in one image, multiple images is also included in the negative prompts.

Give Instructions in English

Image generation models usually generate images closer to your vision when instructed in English rather than other languages.
In this chat app, the chat content is translated into English before extracting words, so it is also compatible with Japanese.

Nova Canvas Chat App Code

Here is the code for the Nova Canvas chat application.

import base64
import json
import os
import random
import boto3
import streamlit as st

REGION = "us-east-1"
IMG_MODEL_ID = "amazon.nova-canvas-v1:0"
TXT_MODEL_ID = "anthropic.claude-3-haiku-20240307-v1:0"

@st.cache_resource
def get_bedrock_client():
    return boto3.client(service_name="bedrock-runtime", region_name=REGION)

def generate_image(native_message, image_size, image_num, system_prompt, ng_text):
    """Generate images"""
    message = system_prompt + native_message
    print(f'textToImageParams: {message}')

    seed = random.randint(0, 858993460)
    native_request = {
        "taskType": "TEXT_IMAGE",
        "textToImageParams": {
            "text": message,
            "negativeText": ng_text
        },
        "imageGenerationConfig": {
            "seed": seed,
            "quality": "standard",
            "height": image_size,
            "width": image_size,
            "cfgScale": 10,
            "numberOfImages": image_num,
        },
    }
    request = json.dumps(native_request)

    bedrock_client = get_bedrock_client()
    response = bedrock_client.invoke_model(modelId=IMG_MODEL_ID, body=request)
    model_response = json.loads(response["body"].read())

    image_path_list = []
    for base64_image_data in model_response["images"]:

        # Save image to local folder
        i, output_dir = 1, "output"
        if not os.path.exists(output_dir):
            os.makedirs(output_dir)
        while os.path.exists(os.path.join(output_dir, f"nova_canvas_{i}.png")):
            i += 1

        image_data = base64.b64decode(base64_image_data)        
        image_path = os.path.join(output_dir, f"nova_canvas_{i}.png")
        with open(image_path, "wb") as file:
            file.write(image_data)

        print(f"The generated image has been saved to {image_path}")
        image_path_list.append(image_path)

    return image_path_list


def textract_from_input(message):
    """Translate Japanese instructions in chat to English and extract key words.
    This process uses foundational models like Haiku or Sonnet for text processing, not Nova Canvas.
    """
    bedrock_client = get_bedrock_client()
    system_prompt = "Translate Other language to English and extract key English words from the translated sentence. "\
    "Extract less than 10 meaningful English words, separated by commas. "\
    "Do not include prepositions or articles. "\
    "<examleInput>かわいい猫と女の子が楽しく遊んでいる</examleInput>"\ # example of Japanese
    "<examleOutput>cute cat, girl, play, joyful</examleOutput>"

    message = {
        "role": "user",
        "content": [{"text": message}]
    }
    messages = [message]
    system_prompts = [{"text" : system_prompt}]

    inference_config = {"temperature": 0.5}
    additional_model_fields = {"top_k": 200}

    response = bedrock_client.converse(
        modelId=TXT_MODEL_ID,
        messages=messages,
        system=system_prompts,
        inferenceConfig=inference_config,
        additionalModelRequestFields=additional_model_fields
    )
    words = response["output"]["message"]["content"][0]["text"]
    return words


def display_history(messages):
    """Display chat history"""
    for message in messages:
        display_img_content(message)


def display_img_content(message):
    """Display message and images
    Handles multiple images generated by Nova
    """
    contents = message["content"]
    print(f'message contents: {contents}')
    with st.chat_message(message["role"]):
        for content in contents:
            if content.get('text', None) != None:
                st.write(content["text"])
            else:
                st.image(content["image"])


def sidebar():
    """Display sidebar"""
    with st.sidebar:
        st.sidebar.title("Image Settings")

        # Image size
        image_size = st.sidebar.slider(
            "Image Size",
            min_value=320,
            max_value=960,
            step=64,
            value=320
        )

        # Number of images
        image_number = st.sidebar.slider(
            "Number of Images",
            min_value=1,
            max_value=5,
            step=1,
            value=1
        )

        # System prompt
        system_prompt = st.sidebar.text_area(
            "System Prompt",
            value="minimalist icon, simple, flat design, dual-line design, 1.5px stroke weight line, "\
            "solid background, monochromatic, 2D, ",
            height=200
        )

        # Negative prompt
        negative_text = st.sidebar.text_area(
            "Negative Prompt",
            value="3D, color, photo, multiple images",
            height=100
        )

        return image_size, image_number, system_prompt, negative_text


def main():
    """Main process"""
    st.title("Simple Icon Generator by Nova Canvas")
    img_size, img_num, system_prompt, negative_text = sidebar()

    if "messages" not in st.session_state:
        st.session_state.messages = []

    display_history(st.session_state.messages)

    if prompt := st.chat_input("What's up?"):
        input_msg = {"role": "user", "content": [{"text": prompt}]}
        display_img_content(input_msg)
        st.session_state.messages.append(input_msg)
        print(f'st.session_state.messages: {st.session_state.messages}')

        # Expand input content
        print(f'input_msg: {input_msg}')
        textract_resp = textract_from_input(input_msg["content"][0]["text"])
        print(f'texttract: {textract_resp}')

        # Generate images
        resp_img_path_list = generate_image(
            textract_resp, img_size, img_num, system_prompt, negative_text
        )

        resp_img_contents = []
        for resp_img_path in resp_img_path_list:
            resp_img_contents.append({"text": f"Image created! filepath: {resp_img_path}"})
            resp_img_contents.append({"image": resp_img_path})
        resp_msg = {
            "role": "assistant",
            "content": resp_img_contents
        }
        display_img_content(resp_msg)
        st.session_state.messages.append(resp_msg)


if __name__ == "__main__":
    main()

3. Running the Application

Let's run the application! Start the application with the following streamlit command.

streamlit run nova_canvas_chat.py --server.port 8080

In Cloud9, you can display the application by opening Preview -> Preview Running Application from the toolbar. The same should apply to local environments like VSCode, VSCode Server, and SageMaker Studio code editor.

If you see a startup screen like this, you have succeeded.

Now, let's instruct the chat to create the icon image you want. This time, I instructed it to "Create an icon for a ToDo list that can manage what I want to do." Since I want multiple suggestions, I set the number of images to 3 from the sidebar.

It created a nice ToDo list icon! Although the image shows only two, three images were generated, allowing you to choose the icon that best matches your image.

The created images are stored in the output folder within the project folder you created.

Extend the Chat App

If you want to distribute this application to many users, deploying it on an ECS container can achieve that. Also, since saving locally is inconvenient, let's use an S3 bucket as the storage destination.

Especially when distributing the application on EC2 or containers, it is easier to manage using file storage like S3 rather than local storage on the server.

Additionally, by naming the image files with creation timestamps or UUID, the chat application can function properly even when used by multiple people.

It might also be good to manage users with Cognito.

Summary

We created an image generation chat application using Amazon Nova Canvas.

Since it was almost my first time generating images, it took some time starting from prompt engineering basics, but the joy of generating an image as intended is great!

Understanding the Difference Between Virtual AZ and Physical AZ Through Failures

Yuki Ogawa — Mon, 28 Apr 2025 13:58:33 +0000

Conclusion

The ap-northeast-1a (Virtual AZ) displayed in subnet settings and the actual data center apne1-az1 (Physical AZ) correspond differently for each AWS account.

What is an Availability Zone (AZ)?

An Availability Zone (AZ) refers to a physical data center managed by AWS. One AZ can consist of one or more data centers.

An Availability Zone (AZ) is one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region. AZs give customers the ability to operate production applications and databases that are more highly available, fault tolerant, and scalable than would be possible from a single data center. All AZs in an AWS Region are interconnected with high-bandwidth, low-latency networking, over fully redundant, dedicated metro fiber providing high-throughput, low-latency networking between AZs. All traffic between AZs is encrypted. The network performance is sufficient to accomplish synchronous replication between AZs. AZs make partitioning applications for high availability easy. If an application is partitioned across AZs, companies are better isolated and protected from issues such as power outages, lightning strikes, tornadoes, earthquakes, and more. AZs are physically separated by a meaningful distance, many kilometers, from any other AZ, although all are within 100 km (60 miles) of each other.

As of 2025, the Tokyo region (ap-northeast-1) has four AZs. The Osaka region (ap-northeast-3) also has three AZs.
https://aws.amazon.com/about-aws/global-infrastructure/regions_az/

In this article, we will refer to an AZ composed of one or more data centers managed by AWS as a Physical AZ.

What is a Virtual AZ?

When creating a subnet, you set the number of AZs like this:

For example, by placing two subnets in a VPC in different AZs, you can ensure redundancy.

In this article, we will refer to the AZs assigned with names like ap-northeast-1a at this time as Virtual AZ.

Incident in the Tokyo Region on April 15, 2025

The following information was obtained from the Health Dashboard. The incident occurred in apne1-az4.

Between 4:40 PM and 5:43 PM JST, there was an issue affecting connectivity to some EC2 instances in a single Availability Zone (apne1-az4) in the AP-NORTHEAST-1 Region. This was due to a power outage affecting both the primary and secondary power sources for the affected EC2 instances. During this time, customers may have experienced increased error rates and latency for instances launched in the affected Availability Zone and for other AWS APIs using the affected EC2 instances. Engineers responded automatically within minutes and immediately began investigating mitigation measures. This issue is not expected to recur. A small number of remaining instances are hosted on hardware affected by the power outage. We continue to work on recovering all affected instances and volumes, but we recommend replacing any remaining affected instances or volumes if possible for immediate recovery. The issue has been resolved, and the service is operating normally.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html

Is Physical AZ NOT Virtual AZ?

Since the incident occurred in apne1-az4, does it mean that ap-northeast-1d and other AZs were not affected?
For example, if security groups were deployed in ap-northeast-1a and ap-northeast-1c, can we say they were not affected?

The mapping of Physical AZs to Virtual AZs varies by AWS account, and the same Virtual AZ name does not necessarily correspond to the same Physical AZ.

https://docs.aws.amazon.com/ram/latest/userguide/working-with-az-ids.html

It looks something like this. Since it varies by account, it needs to be confirmed in each environment.

How to Check the Correspondence Between Virtual AZs and Physical AZs

You can check using the following AWS CLI command:

aws ec2 describe-availability-zones --region us-west-2

The output will look like this. ZoneId represents the Physical AZ.

{
    "AvailabilityZones": [
        {
            "State": "available",
            "OptInStatus": "opt-in-not-required",
            "Messages": [],
            "RegionName": "us-west-2",
            "ZoneName": "us-west-2a",
            "ZoneId": "usw2-az2",
            "GroupName": "us-west-2",
            "NetworkBorderGroup": "us-west-2",
            "ZoneType": "availability-zone"
        },
        {
            "State": "available",
            "OptInStatus": "opt-in-not-required",
            "Messages": [],
            "RegionName": "us-west-2",
            "ZoneName": "us-west-2b",
            "ZoneId": "usw2-az1",
            "GroupName": "us-west-2",
            "NetworkBorderGroup": "us-west-2",
            "ZoneType": "availability-zone"
        },
        {
            "State": "available",
            "OptInStatus": "opt-in-not-required",
            "Messages": [],
            "RegionName": "us-west-2",
            "ZoneName": "us-west-2c",
            "ZoneId": "usw2-az3",
            "GroupName": "us-west-2",
            "NetworkBorderGroup": "us-west-2",
            "ZoneType": "availability-zone"
        },
        {
            "State": "available",
            "OptInStatus": "opt-in-not-required",
            "Messages": [],
            "RegionName": "us-west-2",
            "ZoneName": "us-west-2d",
            "ZoneId": "usw2-az4",
            "GroupName": "us-west-2",
            "NetworkBorderGroup": "us-west-2",
            "ZoneType": "availability-zone"
        }
    ]
}

Importance of Design for Failure

This incident has reinforced the importance of Design for Failure.
It is crucial to design for redundancy across multiple AZs rather than relying on a single AZ configuration.