DocTalk

Prerequisites

1. Install Python along with the app dependencies on your system

   1. Install using these packages using pip: pip install -r requirements.txt

2. Setup a .env file in the root of the project with necessary environment variables, and you will need:

   1. OctoAI API Token
   2. Pinecone API Key

3. The playwrite module is required for the LangChain document loader

   1. Initialize before first use with the following command: python3 -m playwright install

Environment variables

Make sure you have the .env file in the project's app directory, following this template:

PINECONE_API_KEY=YOUR-TOKEN
PINECONE_ENV=gcp-starter
OCTOAI_TOKEN=YOUR-TOKEN
OCTOAI_ENDPOINT_URL="https://text.octoai.run/v1/chat/completions"
OCTOAI_MODEL="llama-2-13b-chat-fp16"

Replace the placeholder values with your actual API keys and endpoints. Do remember to update the PINECONE_ENV variable if you are using an existing Pinecone environment.

Running the app

As a command-line interface (CLI)

To run using CLI, execute the main script: python main.py

You will be prompted to enter the data source and your query. After providing the needed inputs, the application will process the request and display the output. Note: The first time you run the application it takes a bit (several minutes) to populate the pinecone index for the first time. Afterward it should be faster.

As an AWS Lambda function

Package the app with all needed dependencies and upload to AWS Lambda. Set the handler function as main.handler.

Deploying the app to as a container image

• AWS account

• AWS CLI configured

• Docker installed

• AWS SAM CLI installed

To deploy the Python app to AWS Lambda using a container image, follow these steps:

1. Prepare the Dockerfile: make sure it is setup correclty to build a container image for AWS Lambda.

   1. Specify the base image

   2. Copy your app code into the container

   3. Install any dependencies

   4. Set the entry point for your Lambda function

2. Build the container image by using the AWS SAM CLI with the command (generally): sam build --use-container

3. Test locally (optional, but advised) to see if the Lambda function is working using SAM: sam local invoke

To deploy to AWS Lambda follow these steps:

1. Upload the container image to Amazon Elastic Container Registry (ECR)

   1. You can use Docker CLI to do this

2. Use AWS SAM CLI to deploy the app

   1. Package the app and deploy using a SAM template

      1. sam package --output-template-file packaged.yaml --s3-bucket
      2. sam deploy --template-file packaged.yaml --stack-name --capabilities CAPABILITY_IAM

3. Configure Lambda and API Gateway

   1. Do this in the AWS Management Console

   2. Increase the function timeout to 15 minutes

   3. Scale up resource specs — increase the Memory to 2048 MB

Add configuration variables:

You can follow the steps in the AWS API Gateway docs, or use the python script at api-gateway.py to create an API gateway for the Lambda function that allows calls over HTTPS.

Contribute

We would love to have your contributions to this project, as long as the code follows the project's coding standards and includes appropriate testing.