Karakeep and Bedrock with Amazon Nova2 Lite for the perfect bookmarking solution

During my day, I stumble upon a lot of interesting articles, even during work hours. Unfortunately, I cannot always read them instantly, so I used Pocket to have a reading list for my free time.

After Mozilla announced the decommissioning of the service. I started to find an alternative, better if open-source and that can be self-hosted.

After some searching, i found KaraKeep, an excellent replacement for Pocket: it has even more features a first-class mobile application and browser extension. Lucky for me, it also has native S3 integration for saved assets and, most importantly, AI summarization features with AI auto-tagging.

Since I already had a machine running Immich, I hosted it using docker-compose, but it is ready to run even on Kubernetes Clusters. I won’t describe the installation and configuration process, as the project has an extensive documentation

Just for reference, don’t forget to set your S3 endpoint in the .env config file

ASSET_STORE_S3_ENDPOINT=https://s3.eu-west-1.amazonaws.com
ASSET_STORE_S3_REGION=eu-west-1
ASSET_STORE_S3_BUCKET=yourbucketname

After three months of using it, I have to say that it is perfect.

What was a plus for me, it was the auto-tagging feature, and I found the ability to customize the prompt to summarize and categorize saved articles as an awesome addition.

Here are my additions to the already good custom prompt:

I first tested the feature with OpenRouter free models, and, as Amazon Nova2 Lite got out, I wanted to try it.

KaraKeep uses OpenAI-compatible APIs, and Bedrock, through project Mantle, offers a layer of compatibility: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html

Unfortunately, at this time, Amazon Nova2 Lite is not available using project Mantle. You can check model availability with this little python snippet, setting the standard environment variables

• OPENAI_API_KEY – Set to your Amazon Bedrock API key
• OPENAI_BASE_URL – Set to the Amazon Bedrock endpoint for your region (for example, https://bedrock-mantle.eu-west-1.api.aws/v1)

and then running this python snippet

# List all available models using the OpenAI SDK
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables

from openai import OpenAI
client = OpenAI()
models = client.models.list()

for model in models.data:
   print(model.id)

Do we hit a stop? Of course not!

The aws-samples GitHub repos has always something interesting. This time it offers the bedrock-access-gateway solution. Simple to deploy and effective!

      OPENAI_BASE_URL: https://yourapigw.aws-region.amazonaws.com/api/v1
      OPENAI_API_KEY: api_key_in_secrets_manager
      INFERENCE_TEXT_MODEL: eu.amazon.nova-2-lite-v1:0
      INFERENCE_IMAGE_MODEL: eu.amazon.nova-2-lite-v1:0

That’s all, folks!

In addition, you can configure other software that require OpenAI compatible apis to use your custom, private solution.