Managed Database

DuckDB has just launched a new pg_duckdb extension and I want to use it in my PostgreSQL database to improve the performance in analytics. pg_duckdb is a Postgres extension that embeds DuckDB's columnar-vectorized analytics engine and features into Postgres. The GitHub repo: https://github.com/duckdb/pg_duckdb

Why this is important: Our company specializes in AI applications and leverages DigitalOcean for all our AI infrastructure needs. We manage millions of vector records in our DigitalOcean Managed Database, currently utilizing the pgvector extension and Hnsw index type. However, we face significant challenges with our current setup: Our Hnsw indexes are approximately 50GB in size, consuming substantial RAM and making scalability a challenge. Index building is time-intensive, taking several days to complete, which hampers our operational efficiency. As we aim to triple our current indexing capacity to support business growth, it is crucial to adopt more cost-effective and faster technologies. pgvectorscale presents a viable solution that can help us reduce costs and enhance indexing speeds, which are essential for our continued growth and competitiveness. What is pgvectorscale: pgvectorscale is an extension that builds upon pgvector to offer higher performance in embedding search and more cost-efficient storage for AI applications, making it an ideal upgrade for our database systems. It includes key innovations such as: StreamingDiskANN : A new index type inspired by the DiskANN algorithm, which is a research output from Microsoft. Statistical Binary Quantization : Developed by researchers at Timescale, this advanced compression method improves upon traditional Binary Quantization techniques. Additional Information and Resources: Performance Metrics: According to Timescale, pgvectorscale achieves performance up to 28x faster than Pinecone, with significant cost reductions. More details can be found in their recent blog post: Timescale Blog: pgvector is now as fast as Pinecone at 75% less cost Source Code and Documentation: For technical details and implementation specifics, refer to the pgvectorscale GitHub repository: GitHub - pgvectorscale ---- We believe that integrating pgvectorscale into DigitalOcean's managed PostgreSQL offerings will not only benefit our operations but also enhance the service offering for other clients with similar needs. We request your consideration of this feature for a future release.

Hi, This extension is released by Supabase and powers its webhook functionality. It enables asynchronous (non-blocking) HTTPS requests with SQL. We're a DigitalOcean customer and would love to build our app webhooks using this extension. https://github.com/supabase/pg_net Thank you.

The managed Postgres is configured to log everything at the moment which is unnecessary and is expensive when you forward all those logs to a centralized logging platform. We mainly care about connection logging for security reasons and want to disable statement logging. Heroku Postgres, for example, allows configuring statement logging, including none, all or schema changes only. We need the same from DO! Thank you

It's not clear how to do this. It should be easy.

Dear Digital Ocean! In my application the following use case came up, that I would like to do long term data archiving on parts of my operational data in a managed postgreSQL cluster. I found the list of the supported extensions here in the docs . I figured, that I would like to use Parquet files for archiving and analytics in my s3 compatible (do spaces) storage. For that I need the following extension: pg_parquet But sadly this extension is not supported yet. To configure this extension, you have to provide credentials for your object storage , which could be set from a config file or environment variables . That capability to manage some parts of the filesystem or extend envvars alongside the pg_parquet extension support would be really helpful. Hope to hear from this feature request soon! Best Regards, Bendegúz

It would be nice if knowledge bases supported the parque format

There is no point of using unnecessary TLS handshakes etc slowing down performance of apps if we are connecting a managed database from within a secure and trusted VPC.

Apache AGE is a PostgreSQL extension that provides graph database functionality. The goal of Apache AGE is to enable users of Postgres to use graph query modeling in unison with Postgres’ existing relational model. Apache AGE is a Top-Level Project of the Apache Software Foundation since May 2022. It is licensed under the Apache License 2.0.

pganalyze provides a more robust set of utilities for managing queries