Forem: Jayanth MKV

The Dark Side of VLMs: What's Really Going Wrong

Jayanth MKV — Thu, 28 Nov 2024 06:28:58 +0000

Your high-stakes decisions might be based on flawed logic. Here's the scoop:

Systemic Reasoning: The Weak Link
Vision-Language Models (VLMs) are notorious for skipping over systematic thought processes, arriving at answers too quickly and with catastrophic results. Think 9 out of 10 times wrong.

The LLaVA-o1 Revolution
Meet LLaVA-o1, a VLM game-changer. By reason step-by-step, it avoids premature conclusions and verified results:

Stage-level beam search
Inference-time scaling
Iterative reasoning

What makes OpenAI's o1 model so unique?
It breaks down complex problems into bite-sized pieces:

Logical thinking at its finest
Using multiple attempts to reach the correct solution

Four Game-Changing Stages of Reasoning
LLaVA-o1's secret sauce:

Problem Analysis
Hypothesis Generation
Hypothesis Verification
Confidence Assessment

Will VLMs ever be trusted for critical decision-making? It's time to rethink our reliance on these models.

Pushing Python Packages to Artifact Registry Using Cloud Build

Jayanth MKV — Fri, 22 Nov 2024 11:15:53 +0000

Google Artifact Registry is a powerful solution for managing and hosting Python package artifacts in a private, secure, and scalable way. This guide provides a step-by-step walkthrough to push Python package .whl files to the Artifact Registry using Google Cloud Build and a secret (creds) from Google Secret Manager for authentication.

Prerequisites

Artifact Registry Setup:

Create a Python repository in your Artifact Registry:

 gcloud artifacts repositories create python-packages \
   --repository-format=python \
   --location=us-central1 \
   --description="Python packages repository"

Secret Setup:
- Store your key as a secret in Google Secret Manager:
```
 gcloud secrets create creds --data-file=path/to/key.json
```

Grant Cloud Build access to the secret:(Optional, can also be done using IAM)

 gcloud secrets add-iam-policy-binding creds \
   --member="serviceAccount:$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')@cloudbuild.gserviceaccount.com" \
   --role="roles/secretmanager.secretAccessor"

Cloud Build Permissions: Ensure your Cloud Build service account has the necessary permissions to access the Artifact Registry and Secret Manager.

Cloud Build YAML Configuration

Here's the full working cloudbuild.yaml file:

options:
  machineType: E2_HIGHCPU_8
  substitutionOption: ALLOW_LOOSE
  logging: CLOUD_LOGGING_ONLY

steps:
  # Step 1: Access the secret `creds` and save it as `key.json`
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: bash
    args:
      - '-c'
      - |
        gcloud secrets versions access latest --secret=creds > /workspace/key.json

  # Step 2: Configure `.pypirc` with the Artifact Registry credentials
  - name: 'python'
    entrypoint: bash
    args:
      - '-c'
      - |
        cat > ~/.pypirc << EOL
        [distutils]
        index-servers = tower-common-repo

        [tower-common-repo]
        repository: https://us-central1-python.pkg.dev/$PROJECT_ID/python-packages/
        username: _json_key_base64
        password: $(base64 -w0 /workspace/key.json)
        EOL

        # Step 3: Build and upload the Python package
        pip install twine build && \
        python -m build && \
        twine upload --repository tower-common-repo dist/* --verbose

Step-by-Step Explanation

Define Build Options:
- Set the machine type, substitution behavior, and logging options.
- These configurations ensure efficient builds and manageable logs.
Retrieve key.json Secret:
- Use gcloud secrets versions access to fetch the key.json file securely from Secret Manager.
- Save the file to a known location (/workspace/key.json).
Configure .pypirc:
- Generate a .pypirc file dynamically. This file is required for twine to authenticate with the Artifact Registry.
- The password is base64-encoded content of key.json.
Build and Push Package:
- Install necessary tools (twine, build).
- Build the Python package (python -m build).
- Use twine upload to push the .whl file to the Artifact Registry.

Triggering the Build

Save the cloudbuild.yaml file and trigger the build or can connect to github repository:

gcloud builds submit --config=cloudbuild.yaml .

Key Points

Secure Secrets Management: The secret (key.json) is accessed securely using Google Secret Manager.
Dynamic Configuration: .pypirc is generated during the build, ensuring no sensitive data is stored in the repository.
Automated Upload: The process automates package building and pushing, reducing manual intervention.

Validation

After the build completes:

Verify the uploaded package in the Artifact Registry:

   gcloud artifacts packages list --repository=python-packages --location=us-central1

Check for errors or warnings in the build logs.

How to Create a Cloud Build to Allow Docker to Download Python Packages from Artifact Registry

Jayanth MKV — Fri, 22 Nov 2024 09:12:10 +0000

Google Cloud's Artifact Registry is a powerful tool for managing your application's dependencies. This guide demonstrates how to create a Cloud Build pipeline to enable Docker to access Python packages stored in Artifact Registry. By following these steps, you can securely manage dependencies and streamline deployments.

Prerequisites

Google Cloud Project: Ensure you have a GCP project set up.
Artifact Registry: A Python repository should already be configured in the Artifact Registry.
Cloud Build: Enable the Cloud Build API for your project.
Authentication: Configure service account permissions to access the Artifact Registry.

Steps to Configure Cloud Build

1. Generate an Artifact Registry Token

Use gcloud auth to generate an access token that will allow the Docker build process to authenticate with the Artifact Registry. Here's how you can do this:

steps:
  # Generate Artifact Registry token
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: bash
    args:
      - '-c'
      - |
        art=$(gcloud auth print-access-token)
        echo "$art" > /workspace/artifact_registry_token
        echo "$art"

2. Use the Token in Docker Build

Once the token is generated, it can be passed to the docker build process as a build argument. Here's how:

  - name: gcr.io/cloud-builders/docker
    id: Build
    env:
      - 'btf=/workspace/artifact_registry_token'
    entrypoint: bash
    args:
      - '-c'
      - |
        docker build \
          --build-arg ARTIFACT_REGISTRY_TOKEN=$(cat $btf) \
          --build-arg PROJECT_ID=$PROJECT_ID \
          -t test-image:latest \
          -f Dockerfile .

3. Create the Dockerfile

The Dockerfile is configured to use the token to download Python packages from Artifact Registry:

# syntax=docker/dockerfile:1

FROM python:3.11-slim

ARG ARTIFACT_REGISTRY_TOKEN
ARG PROJECT_ID

# Keeps Python from buffering stdout and stderr
ENV PYTHONUNBUFFERED=1

WORKDIR /app

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

# Install dependencies using the token
RUN pip install \
    --index-url https://pypi.org/simple \
    --extra-index-url https://oauth2accesstoken:${ARTIFACT_REGISTRY_TOKEN}@us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/simple/ \
    "your-package-name==your-package-version"

# Expose the application port
EXPOSE 8080

# Command to run the application
CMD ["uvicorn", "main:app", "--port=8080", "--host=0.0.0.0"]

4. Add Build Config Options

Finally, define other configurations such as machine type, logging, and substitutions:

options:
  machineType: E2_HIGHCPU_8
  substitutionOption: ALLOW_LOOSE
  logging: CLOUD_LOGGING_ONLY
substitutions:
  _PACKAGE: your-package-name==your-package-version
  _REPOSITORY: python-packages
  _LOCATION: us-central1
  _PROJECT_ID: your-project-id

Tags and Metadata

To organize your builds better, include meaningful tags:

tags:
  - gcp-cloud-build
  - artifact-registry
  - docker-python-packages

Summary

This setup ensures that your Docker builds in Cloud Build can securely pull Python dependencies from your Artifact Registry using an access token. Adjust the provided configuration to your project-specific details, such as package names, repository URLs, and deployment targets.

Implementing this pipeline will improve security and make dependency management seamless for your projects.

GCP publish python package in production

Jayanth MKV — Tue, 19 Nov 2024 15:31:20 +0000

GCP: Publish Python Package in Production

This guide explains how to use Google Artifact Registry to manage shared Python code as a package. This approach eliminates code duplication between your Cloud Functions and server.

Step 1: Structure Your Shared Code

Create a new Python package for your shared logic (e.g., common_logic).

common_logic/
├── setup.py
├── common_logic/
│   ├── __init__.py

Step 2: Create `setup.py`

Define your package configuration in a setup.py file:

from setuptools import setup, find_packages

setup(
    name="common_logic",
    version="0.1.0",
    packages=find_packages(),
    install_requires=[
        "pandas>=1.3.0",
    ],
    author="Your Name",
    author_email="your.email@example.com",
    description="Common logic for app",
)

Step 3: Set Up Google Artifact Registry

Enable the Artifact Registry API:

   gcloud services enable artifactregistry.googleapis.com

Create a Python repository:

   gcloud artifacts repositories create python-packages \
       --repository-format=python \
       --location=us-central1 \
       --description="Python packages repository"

Step 4: Configure Authentication

Create a service account:

   gcloud iam service-accounts create artifact-publisher \
       --description="Service account for publishing to Artifact Registry"

Grant necessary permissions:

   gcloud artifacts repositories add-iam-policy-binding python-packages \
       --location=us-central1 \
       --member="serviceAccount:artifact-publisher@${PROJECT_ID}.iam.gserviceaccount.com" \
       --role="roles/artifactregistry.writer"

Create and download a key:

   gcloud iam service-accounts keys create key.json \
       --iam-account=artifact-publisher@${PROJECT_ID}.iam.gserviceaccount.com

Step 5: Build and Upload Package

Install build tools:

   pip install build twine

Build the package:

   python -m build

Configure twine for Artifact Registry:

   cat > ~/.pypirc << EOL
   [distutils]
   index-servers = common-logic-repo
   [common-logic-repo]
   repository: https://us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/
   username: _json_key_base64
   password: $(base64 -w0 key.json)
   EOL

Upload the package:

   twine upload --repository common-logic-repo dist/*

Step 6: Use the Package

In Cloud Functions

Create a requirements.txt file:

   --index-url https://pypi.org/simple
   --extra-index-url https://oauth2accesstoken:${ARTIFACT_REGISTRY_TOKEN}@us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/simple/
   common-logic==0.1.0

Use the package in your Cloud Function:

   from common_logic import ...

   def cloud_function(request):
       # Your cloud function code using the imported functions
       pass

In Server Code

Add to your server's requirements.txt:

   --index-url https://pypi.org/simple
   --extra-index-url https://oauth2accesstoken:${ARTIFACT_REGISTRY_TOKEN}@us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/simple/
   common-logic==0.1.0

Use it in your server code:

   from common_logic import ...
   # Your server code using the imported functions

Step 7: CI/CD Integration

Add the service account key as a secret in your GitHub repository.
Update your Cloud Build configuration:

   steps:
     - name: 'python'
       entrypoint: pip
       args: ['install', 'build', 'twine']

     - name: 'python'
       entrypoint: python
       args: ['-m', 'build']
       dir: 'common_logic'

     - name: 'python'
       entrypoint: twine
       args: ['upload', '--repository', 'common-logic-repo', 'dist/*']
       dir: 'common_logic'
       env:
         - 'TWINE_USERNAME=_json_key_base64'
         - 'TWINE_PASSWORD=${_ARTIFACT_REGISTRY_KEY}'

Step 8: Version Management

Update the version in setup.py.
Build and upload the new version.
Update requirements.txt in both Cloud Functions and server code.
Deploy both components.

Best Practices

Use semantic versioning for your package.
Pin specific versions in requirements.txt.
Test new versions thoroughly before deploying.
Keep a changelog of version changes.
Use environment variables for PROJECT_ID and LOCATION.
Include comprehensive documentation in your package.

Common Issues and Solutions

Authentication Errors

Verify service account permissions.
Ensure key.json is properly encoded.
Check .pypirc configuration.

Package Not Found

Verify repository URL format.
Check if the package was successfully uploaded.
Ensure requirements.txt uses the correct URL format.

Version Conflicts

Pin specific versions of dependencies.
Use virtual environments for testing.
Document dependency requirements clearly.

Forem: Jayanth MKV

The Dark Side of VLMs: What's Really Going Wrong

Pushing Python Packages to Artifact Registry Using Cloud Build

Prerequisites

Cloud Build YAML Configuration

Step-by-Step Explanation

Triggering the Build

Key Points

Validation

How to Create a Cloud Build to Allow Docker to Download Python Packages from Artifact Registry

Prerequisites

Steps to Configure Cloud Build

1. Generate an Artifact Registry Token

2. Use the Token in Docker Build

3. Create the Dockerfile

4. Add Build Config Options

Tags and Metadata

Summary

GCP publish python package in production

GCP: Publish Python Package in Production

Step 1: Structure Your Shared Code

Step 2: Create setup.py

Step 3: Set Up Google Artifact Registry

Step 4: Configure Authentication

Step 5: Build and Upload Package

Step 6: Use the Package

In Cloud Functions

In Server Code

Step 7: CI/CD Integration

Step 8: Version Management

Best Practices

Common Issues and Solutions

Authentication Errors

Package Not Found

Version Conflicts

Step 2: Create `setup.py`