ENH: Create an interface from lower level C-API to C++ with same APIs as NumPy

[Qjbtiger]

Proposed new feature or change:

Hi!

Recently, I have been diving into a C++ project which needs some matrix operations. Although some matrix libraries like Eigen, are really excellent, I have to spend extra time to get familiar with the grammar of new libraries. Other libraries like NumCpp, which has the same APIs as NumPy, seem to perform less efficiently than NumPy (I know NumPy did many optimizations for matrix operations).

As NumPy is based on lower-level C-API, I am wondering whether it is possible to build an interface from lower-level C-API to C++ API (exposed to users, same APIs as NumPy). Note that it is just an interface from C to C++ and we don't need to implement specific low-level operations.

[seberg]

TL;DR: I think it is a lot of work for little pay-off to be honest and while I might be convinced if someone dedicated a lot of time on it, I don't think NumPy should prioritize it.

A few points:

1. C++ ABI is (to my understanding) never stable, so you need to be headers only pretty much if you ship it with NumPy for Python. Libraries that do this type of merge otherwise have very strict version compatibility requirements.
2. NumPy is a Python module with a lot of history, you won't be able to untangle it completely unless you wedge a new layer in between everywhere which likely will get very messy.
3. You don't want to use Python in a C++ program, but NumPy is Python through and through.

an interface from lower-level C-API to C++ API

Maybe, I am not convinced it would help you much. I might consider explicit C++ API for some parts of NumPy, but that won't be what you are looking for at all.

---

I realize that it can work well for projects to do this dance, but I think the projects that do it, do it because they need to expose C/C++ and Python in a single project/team. Doing this NumPy would be a ton of work/reorganization and in the end it may just be a worse eigen (or whichever).

There is recurring talk about sharing some of the lowest core implementation but of course many are already "shared" (matrix operations are blas/lapack, fft is not NumPy's really, etc.).
Many numeric routines are "hand-optimized" in NumPy and it might make sense to make them sharable (using highway in NumPy may do that to some degree). But it isn't what you want anyway.

[ianthomas23]

@Qjbtiger: xtensor and xtensor-python might provide what you are looking for.