Convert Parquet to Arrow

Convert Parquet to Arrow in seconds with this free online Parquet to Arrow converter

Accepts parquet


Parquet

Apache Parquet (.parquet) is a format that was designed for storing tabular data on disk. It was designed based on the format used in Google's Dremel paper (Dremel later became Big Query).

Parquet files store data in a binary format, which means that they can be efficiently read by computers but are difficult for people to read.

Parquet files have a schema, so means that every value in a column must have the same type. The schema makes Parquet files easier to analyse than CSV files and also helps them to have better compression so they are smaller on disk.

Arrow

Apache Arrow (.arrow) is a format that was designed for storing tabular data in memory (RAM). It was not designed for storing data as files on disk.

Arrow enforces a schema. Which means that every value in a column must have the same value.

Arrow was designed to work well between different data analysis systems without needing to be serialised or deserialized.

Arrow is a binary data format, which means that it can be easily read by computers. But it cannot be read by people. The easiest way to view arrow data is to convert it to CSV first.

It can be best to convert Arrow data to Parquet before saving to disk. Arrow was not designed for storing data to disk, so can be inefficient and slow to query.


Supercharge your data exploration

Open csv, parquet, arrow, json and tsv files straight from your desktop

Or


Share and embed

Share your graphs and data sets.

Share your graphs and data sets. Or embed them directly into web pages.


Work straight from Google Drive

Open csv, parquet, arrow, json and tsv files directly from Drive, Gmail and Classroom by installing the Google Workspace App


How to Convert Parquet to Arrow

Viewing converted data
  1. Select your input Parquet file
  2. Your Parquet file will be converted to Arrow
  3. Download your Arrow file
  4. Click the View button to view your file

How to Convert Parquet to Arrow in Python

We can convert Parquet to Arrow in Python using Pandas or DuckDB

How to Convert Parquet to Arrow using Pandas

First, we need to install pandas

pip install pandas

Then we can load the Parquet file into a dataframe

df = pd.read_parquet('path/to/file.parquet')

Finally, we can export the dataframe to the Arrow format

df.to_feather('path/to/file.arrow', index=False)

How to Convert Parquet to Arrow using DuckDB

First, we need to install duckdb for Python

pip install duckdb

The following duckdb query will copy the contents of a single Parquet file to a Arrow file

duckdb.sql("""COPY (select * from 'path/to/file.parquet') TO 'path/to/file.arrow' (FORMAT 'arrow')""")

If you have more than one Parquet file with the same schema (e.g. your Parquet files are partitioned) then you can use the following

duckdb.sql("""COPY (select * from read_parquet(['path/to/file1.parquet', 'path/to/file2.parquet'])) TO 'path/to/file.arrow' (FORMAT 'arrow')""")