ability to make snapshot of index into new index ?

[gst]

Hi,

at my work ($$$) we would like to be able to make snapshot(s) of a particular index and to put that in a new index name basically (with the date of the snapshot so to be clear).

that is we would like to do something like :

devpi use http://localhost:3141
devpi login root
# then
devpi snapshot root/some-index root/some-index-YYYYMMDD

and the last one would make a snapshot of the root/some-index basically into root/some-index-YYYMMDD.

and the snapshot would obviously use whatever is best to not duplicate the data and do that almost instantaneously (I mean if there is no data duplicate it should be pretty fast).

would that be easily possible ? Obviously the snapshot index would not be updated once created : the same versions of any package available in it would always be available but no more.

I'm trying to look into the code right now but it does not look that easy/immediate to see where/how I could do the change..

[fschulze]

I thought about something like that before. I would name it copy though, as it would be a regular new index and there is no relation to the original index and no restore functionality.

It is not particularly hard to do, but the server side API needs some thought. WebDAV has a COPY HTTP method which might be applicable.

Currently the copy can't be done instantly, as files are stored per index. I plan to change that for a future devpi-server version, but that change is far from trivial, as we do not have a fully relational DB and multiple references to the same file need to be handled properly. There are several ways to do that with varying pros and cons that have to be considered.

Both devpi-client and devpi-server should have all the necessary hooks to implement this as a plugin first. Take a look at devpi-lockdown and devpi-tokens for examples.

[gst]

Currently the copy can't be done instantly, as files are stored per index.

well. unless we use hard-link(s) copy that is, for instance. it's not instantly but it's quite fast (unless you have 100k+ projects).

I tried this way first actually :

# cp -r -l path/to/some-index path/to/some-index-YYYYMMDD
# create of the index
# devpi-clear-search-index
# restart of devpi-server and multiple tries of everything possible

in lot of different order. but nothing triggers a "rescan" of the newly created some-index-YYYYMMDD that is.

I will look more deeply into the plugins..

[fschulze]

The files are referenced from the database. The search index is only for search inside the web interface. So a simple copy on the file system alone does nothing, the data in the database needs to be updated accordingly.

[gst]

So a simple copy on the file system alone does nothing, the data in the database needs to be updated accordingly.

effectively :) thus my issue ;)

[gst]

I'm trying something here : #900