293 lines
11 KiB
Text
293 lines
11 KiB
Text
|
Metadata-Version: 2.4
|
||
|
Name: narwhals
|
||
|
Version: 2.0.1
|
||
|
Summary: Extremely lightweight compatibility layer between dataframe libraries
|
||
|
Project-URL: Homepage, https://github.com/narwhals-dev/narwhals
|
||
|
Project-URL: Documentation, https://narwhals-dev.github.io/narwhals/
|
||
|
Project-URL: Repository, https://github.com/narwhals-dev/narwhals
|
||
|
Project-URL: Bug Tracker, https://github.com/narwhals-dev/narwhals/issues
|
||
|
Author-email: Marco Gorelli <hello_narwhals@proton.me>
|
||
|
License-File: LICENSE.md
|
||
|
Keywords: cudf,dask,dataframes,interoperability,modin,pandas,polars,pyarrow
|
||
|
Classifier: License :: OSI Approved :: MIT License
|
||
|
Classifier: Operating System :: OS Independent
|
||
|
Classifier: Programming Language :: Python
|
||
|
Classifier: Typing :: Typed
|
||
|
Requires-Python: >=3.9
|
||
|
Provides-Extra: cudf
|
||
|
Requires-Dist: cudf>=24.10.0; extra == 'cudf'
|
||
|
Provides-Extra: dask
|
||
|
Requires-Dist: dask[dataframe]>=2024.8; extra == 'dask'
|
||
|
Provides-Extra: duckdb
|
||
|
Requires-Dist: duckdb>=1.0; extra == 'duckdb'
|
||
|
Provides-Extra: ibis
|
||
|
Requires-Dist: ibis-framework>=6.0.0; extra == 'ibis'
|
||
|
Requires-Dist: packaging; extra == 'ibis'
|
||
|
Requires-Dist: pyarrow-hotfix; extra == 'ibis'
|
||
|
Requires-Dist: rich; extra == 'ibis'
|
||
|
Provides-Extra: modin
|
||
|
Requires-Dist: modin; extra == 'modin'
|
||
|
Provides-Extra: pandas
|
||
|
Requires-Dist: pandas>=1.1.3; extra == 'pandas'
|
||
|
Provides-Extra: polars
|
||
|
Requires-Dist: polars>=0.20.4; extra == 'polars'
|
||
|
Provides-Extra: pyarrow
|
||
|
Requires-Dist: pyarrow>=13.0.0; extra == 'pyarrow'
|
||
|
Provides-Extra: pyspark
|
||
|
Requires-Dist: pyspark>=3.5.0; extra == 'pyspark'
|
||
|
Provides-Extra: pyspark-connect
|
||
|
Requires-Dist: pyspark[connect]>=3.5.0; extra == 'pyspark-connect'
|
||
|
Provides-Extra: sqlframe
|
||
|
Requires-Dist: sqlframe>=3.22.0; extra == 'sqlframe'
|
||
|
Description-Content-Type: text/markdown
|
||
|
|
||
|
# Narwhals
|
||
|
|
||
|
<h1 align="center">
|
||
|
<img
|
||
|
width="400"
|
||
|
alt="narwhals_small"
|
||
|
src="https://github.com/narwhals-dev/narwhals/assets/33491632/26be901e-5383-49f2-9fbd-5c97b7696f27">
|
||
|
</h1>
|
||
|
|
||
|
[](https://badge.fury.io/py/narwhals)
|
||
|
[](https://pepy.tech/project/narwhals)
|
||
|
[](https://peps.python.org/pep-0740/)
|
||
|
[](https://pypi.org/project/narwhals)
|
||
|
|
||
|
Extremely lightweight and extensible compatibility layer between dataframe libraries!
|
||
|
|
||
|
- **Full API support**: cuDF, Modin, pandas, Polars, PyArrow.
|
||
|
- **Lazy-only support**: Dask, DuckDB, Ibis, PySpark, SQLFrame. Work in progress: Daft.
|
||
|
|
||
|
Seamlessly support all, without depending on any!
|
||
|
|
||
|
- ✅ **Just use** [a subset of **the Polars API**](https://narwhals-dev.github.io/narwhals/api-reference/), no need to learn anything new
|
||
|
- ✅ **Zero dependencies**, Narwhals only uses what
|
||
|
the user passes in so your library can stay lightweight
|
||
|
- ✅ Separate **lazy** and eager APIs, use **expressions**
|
||
|
- ✅ Support pandas' complicated type system and index, without
|
||
|
either getting in the way
|
||
|
- ✅ **100% branch coverage**, tested against pandas and Polars nightly builds
|
||
|
- ✅ **Negligible overhead**, see [overhead](https://narwhals-dev.github.io/narwhals/overhead/)
|
||
|
- ✅ Let your IDE help you thanks to **full static typing**, see [typing](https://narwhals-dev.github.io/narwhals/api-reference/typing/)
|
||
|
- ✅ **Perfect backwards compatibility policy**,
|
||
|
see [stable api](https://narwhals-dev.github.io/narwhals/backcompat/) for how to opt-in
|
||
|
|
||
|
Get started!
|
||
|
|
||
|
- [Read the documentation](https://narwhals-dev.github.io/narwhals/)
|
||
|
- [Chat with us on Discord!](https://discord.gg/V3PqtB4VA4)
|
||
|
- [Join our community call](https://calendar.google.com/calendar/embed?src=27ff6dc5f598c1d94c1f6e627a1aaae680e2fac88f848bda1f2c7946ae74d5ab%40group.calendar.google.com)
|
||
|
- [Read the contributing guide](https://github.com/narwhals-dev/narwhals/blob/main/CONTRIBUTING.md)
|
||
|
|
||
|
<details>
|
||
|
<summary>Table of contents</summary>
|
||
|
|
||
|
- [Narwhals](#narwhals)
|
||
|
- [Installation](#installation)
|
||
|
- [Usage](#usage)
|
||
|
- [Example](#example)
|
||
|
- [Scope](#scope)
|
||
|
- [Roadmap](#roadmap)
|
||
|
- [Used by](#used-by)
|
||
|
- [Sponsors and institutional partners](#sponsors-and-institutional-partners)
|
||
|
- [Appears on](#appears-on)
|
||
|
- [Why "Narwhals"?](#why-narwhals)
|
||
|
|
||
|
</details>
|
||
|
|
||
|
## Installation
|
||
|
|
||
|
- pip (recommended, as it's the most up-to-date)
|
||
|
```
|
||
|
pip install narwhals
|
||
|
```
|
||
|
- conda-forge (also fine, but the latest version may take longer to appear)
|
||
|
```
|
||
|
conda install -c conda-forge narwhals
|
||
|
```
|
||
|
|
||
|
## Usage
|
||
|
|
||
|
There are three steps to writing dataframe-agnostic code using Narwhals:
|
||
|
|
||
|
1. use `narwhals.from_native` to wrap a pandas/Polars/Modin/cuDF/PyArrow
|
||
|
DataFrame/LazyFrame in a Narwhals class
|
||
|
2. use the [subset of the Polars API supported by Narwhals](https://narwhals-dev.github.io/narwhals/api-reference/)
|
||
|
3. use `narwhals.to_native` to return an object to the user in its original
|
||
|
dataframe flavour. For example:
|
||
|
|
||
|
- if you started with pandas, you'll get pandas back
|
||
|
- if you started with Polars, you'll get Polars back
|
||
|
- if you started with Modin, you'll get Modin back (and compute will be distributed)
|
||
|
- if you started with cuDF, you'll get cuDF back (and compute will happen on GPU)
|
||
|
- if you started with PyArrow, you'll get PyArrow back
|
||
|
|
||
|
<h1 align="left">
|
||
|
<img
|
||
|
width="600"
|
||
|
alt="narwhals_gif"
|
||
|
src="https://github.com/user-attachments/assets/88292d3c-6359-4155-973d-d0f8e3fbf5ac">
|
||
|
|
||
|
</h1>
|
||
|
|
||
|
## Example
|
||
|
|
||
|
Narwhals allows you to define dataframe-agnostic functions. For example:
|
||
|
|
||
|
```python
|
||
|
import narwhals as nw
|
||
|
from narwhals.typing import IntoFrameT
|
||
|
|
||
|
|
||
|
def agnostic_function(
|
||
|
df_native: IntoFrameT,
|
||
|
date_column: str,
|
||
|
price_column: str,
|
||
|
) -> IntoFrameT:
|
||
|
return (
|
||
|
nw.from_native(df_native)
|
||
|
.group_by(nw.col(date_column).dt.truncate("1mo"))
|
||
|
.agg(nw.col(price_column).mean())
|
||
|
.sort(date_column)
|
||
|
.to_native()
|
||
|
)
|
||
|
```
|
||
|
|
||
|
You can then pass `pandas.DataFrame`, `polars.DataFrame`, `polars.LazyFrame`, `duckdb.DuckDBPyRelation`,
|
||
|
`pyspark.sql.DataFrame`, `pyarrow.Table`, and more, to `agnostic_function`. In each case, no additional
|
||
|
dependencies will be required, and computation will stay native to the input library:
|
||
|
|
||
|
```python
|
||
|
import pandas as pd
|
||
|
import polars as pl
|
||
|
from datetime import datetime
|
||
|
|
||
|
data = {
|
||
|
"date": [datetime(2020, 1, 1), datetime(2020, 1, 8), datetime(2020, 2, 3)],
|
||
|
"price": [1, 4, 3],
|
||
|
}
|
||
|
print("pandas result:")
|
||
|
print(agnostic_function(pd.DataFrame(data), "date", "price"))
|
||
|
print()
|
||
|
print("Polars result:")
|
||
|
print(agnostic_function(pl.DataFrame(data), "date", "price"))
|
||
|
```
|
||
|
|
||
|
```terminal
|
||
|
pandas result:
|
||
|
date price
|
||
|
0 2020-01-01 2.5
|
||
|
1 2020-02-01 3.0
|
||
|
|
||
|
Polars result:
|
||
|
shape: (2, 2)
|
||
|
┌─────────────────────┬───────┐
|
||
|
│ date ┆ price │
|
||
|
│ --- ┆ --- │
|
||
|
│ datetime[μs] ┆ f64 │
|
||
|
╞═════════════════════╪═══════╡
|
||
|
│ 2020-01-01 00:00:00 ┆ 2.5 │
|
||
|
│ 2020-02-01 00:00:00 ┆ 3.0 │
|
||
|
└─────────────────────┴───────┘
|
||
|
```
|
||
|
|
||
|
See the [tutorial](https://narwhals-dev.github.io/narwhals/basics/dataframe/) for several examples!
|
||
|
|
||
|
## Scope
|
||
|
|
||
|
- Do you maintain a dataframe-consuming library?
|
||
|
- Do you have a specific Polars function in mind that you would like Narwhals to have in order to make your work easier?
|
||
|
|
||
|
If you said yes to both, we'd love to hear from you!
|
||
|
|
||
|
## Roadmap
|
||
|
|
||
|
See [roadmap discussion on GitHub](https://github.com/narwhals-dev/narwhals/discussions/1370)
|
||
|
for an up-to-date plan of future work.
|
||
|
|
||
|
## Used by
|
||
|
|
||
|
Join the party!
|
||
|
|
||
|
- [altair](https://github.com/vega/altair/)
|
||
|
- [bokeh](https://github.com/bokeh/bokeh)
|
||
|
- [darts](https://github.com/unit8co/darts)
|
||
|
- [formulaic](https://github.com/matthewwardrop/formulaic)
|
||
|
- [gt-extras](https://github.com/posit-dev/gt-extras)
|
||
|
- [hierarchicalforecast](https://github.com/Nixtla/hierarchicalforecast)
|
||
|
- [marimo](https://github.com/marimo-team/marimo)
|
||
|
- [metalearners](https://github.com/Quantco/metalearners)
|
||
|
- [mosaic](https://github.com/uwdata/mosaic)
|
||
|
- [panel-graphic-walker](https://github.com/panel-extensions/panel-graphic-walker)
|
||
|
- [plotly](https://plotly.com)
|
||
|
- [pointblank](https://github.com/posit-dev/pointblank)
|
||
|
- [pymarginaleffects](https://github.com/vincentarelbundock/pymarginaleffects)
|
||
|
- [py-shiny](https://github.com/posit-dev/py-shiny)
|
||
|
- [rio](https://github.com/rio-labs/rio)
|
||
|
- [scikit-lego](https://github.com/koaning/scikit-lego)
|
||
|
- [scikit-playtime](https://github.com/koaning/scikit-playtime)
|
||
|
- [tabmat](https://github.com/Quantco/tabmat)
|
||
|
- [tea-tasting](https://github.com/e10v/tea-tasting)
|
||
|
- [timebasedcv](https://github.com/FBruzzesi/timebasedcv)
|
||
|
- [tubular](https://github.com/lvgig/tubular)
|
||
|
- [Validoopsie](https://github.com/akmalsoliev/Validoopsie)
|
||
|
- [vegafusion](https://github.com/vega/vegafusion)
|
||
|
- [wimsey](https://github.com/benrutter/wimsey)
|
||
|
|
||
|
Feel free to add your project to the list if it's missing, and/or
|
||
|
[chat with us on Discord](https://discord.gg/V3PqtB4VA4) if you'd like any support.
|
||
|
|
||
|
## Sponsors and institutional partners
|
||
|
|
||
|
Narwhals is 100% independent, community-driven, and community-owned.
|
||
|
We are extremely grateful to the following organisations for having
|
||
|
provided some funding / development time:
|
||
|
|
||
|
- [Quansight Labs](https://labs.quansight.org)
|
||
|
- [Quansight Futures](https://www.qi.ventures)
|
||
|
- [OpenTeams](https://www.openteams.com)
|
||
|
- [POSSEE initiative](https://possee.org)
|
||
|
- [BYU-Idaho](https://www.byui.edu)
|
||
|
|
||
|
If you contribute to Narwhals on your organization's time, please let us know. We'd be happy to add your employer
|
||
|
to this list!
|
||
|
|
||
|
## Appears on
|
||
|
|
||
|
Narwhals has been featured in several talks, podcasts, and blog posts:
|
||
|
|
||
|
- [Talk Python to me Podcast](https://youtu.be/FSH7BZ0tuE0)
|
||
|
Ahoy, Narwhals are bridging the data science APIs
|
||
|
|
||
|
- [Python Bytes Podcast](https://www.youtube.com/live/N7w_ESVW40I?si=y-wN1uCsAuJOKlOT&t=382)
|
||
|
Episode 402, topic #2
|
||
|
|
||
|
- [Super Data Science: ML & AI Podcast](https://www.youtube.com/watch?v=TeG4U8R0U8U)
|
||
|
Narwhals: For Pandas-to-Polars DataFrame Compatibility
|
||
|
|
||
|
- [Sample Space Podcast | probabl](https://youtu.be/8hYdq4sWbbQ?si=WG0QP1CZ6gkFf18b)
|
||
|
How Narwhals has many end users ... that never use it directly. - Marco Gorelli
|
||
|
|
||
|
- [The Real Python Podcast](https://www.youtube.com/watch?v=w5DFZbFYzCM)
|
||
|
Narwhals: Expanding DataFrame Compatibility Between Libraries
|
||
|
|
||
|
- [Pycon Lithuania](https://www.youtube.com/watch?v=-mdx7Cn6_6E)
|
||
|
Marco Gorelli - DataFrame interoperatiblity - what's been achieved, and what comes next?
|
||
|
|
||
|
- [Pycon Italy](https://www.youtube.com/watch?v=3IqUli9XsmQ)
|
||
|
How you can write a dataframe-agnostic library - Marco Gorelli
|
||
|
|
||
|
- [Polars Blog Post](https://pola.rs/posts/lightweight_plotting/)
|
||
|
Polars has a new lightweight plotting backend
|
||
|
|
||
|
- [Quansight Labs blog post (w/ Scikit-Lego)](https://labs.quansight.org/blog/scikit-lego-narwhals)
|
||
|
How Narwhals and scikit-lego came together to achieve dataframe-agnosticism
|
||
|
|
||
|
## Why "Narwhals"?
|
||
|
|
||
|
[Coz they are so awesome](https://youtu.be/ykwqXuMPsoc?si=A-i8LdR38teYsos4).
|
||
|
|
||
|
Thanks to [Olha Urdeichuk](https://www.fiverr.com/olhaurdeichuk) for the illustration!
|