A fledgling file format that aims to address limitations in the widely-used Parquet is under review for adoption by an open source foundation.

Lance is built on the idea that Parquet – widely used in AWS, Azure, and Google data lakes – shows its age when it comes to machine learning and AI, and an additional, complementary format better suits those requirements.

Behind the format is Chang She, one of the original contributors to the pandas software library for data manipulation and analysis, who is now CEO and co-founder of LanceDB, which supports and develops the format.

"In 2022, we had our first Lance 0.01 release, we were widely seen as a little bit crazy for suggesting that there was a better alternative to Parquet. Certainly, the world has changed since then," She said.

The turni

See Full Page