[venv] Implications of pip download behavior change

[Diego Costantini]

Hi,
on pip version 8 (I know we are at 9 now :) ) I started getting deprecation
warnings when using "pip install --download".

So I did what suggested and moved to "pip download".

This change did not preserve the behavior.

Before I was getting all whl dependencies downloaded, now some of them are
tar.gz.

pip install --download=../lib "$SRC_PYTHON_PACKAGE"

Collecting fastdtw==0.3.0 (from myPackage)
Saved /path/fastdtw-0.3.0-cp27-none-any.whl


pip download -d ../lib "$SRC_PYTHON_PACKAGE"

Collecting fastdtw==0.3.0 (from myPackage)
Using cached fastdtw-0.3.0.tar.gz
Saved /path/fastdtw-0.3.0.tar.gz

Why did this happen and what are the implications of having a tar.gz vs whl
(I did not find anything conclusive in my search so far)?

Some further info. This happens for:
1 - deps built by me as tar.gz (how did they end up in my folder as whl
before?)
2 - 3rd party deps
3 - Looking for the cache, I cannot find the tar.gz in my system

[Paul Moore]

A quick check of PyPI doesn't show a wheel for fastdtw 0.3.0. Have you
perhaps got an extra index configured somewhere? And if so, did you
only configure it for the "install" command, and not for the
"download" command?

On 19 January 2017 at 16:11, Diego Costantini

Post by Diego Costantini
Hi,
on pip version 8 (I know we are at 9 now :) ) I started getting deprecation
warnings when using "pip install --download".
So I did what suggested and moved to "pip download".
This change did not preserve the behavior.
Before I was getting all whl dependencies downloaded, now some of them are
tar.gz.
pip install --download=../lib "$SRC_PYTHON_PACKAGE"
Collecting fastdtw==0.3.0 (from myPackage)
Saved /path/fastdtw-0.3.0-cp27-none-any.whl
pip download -d ../lib "$SRC_PYTHON_PACKAGE"
Collecting fastdtw==0.3.0 (from myPackage)
Using cached fastdtw-0.3.0.tar.gz
Saved /path/fastdtw-0.3.0.tar.gz
Why did this happen and what are the implications of having a tar.gz vs whl
(I did not find anything conclusive in my search so far)?
1 - deps built by me as tar.gz (how did they end up in my folder as whl
before?)
2 - 3rd party deps
3 - Looking for the cache, I cannot find the tar.gz in my system
--
You received this message because you are subscribed to the Google Groups
"virtualenv" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/python-virtualenv.
For more options, visit https://groups.google.com/d/optout.

...

[Diego Costantini]

You might be onto something here :)

I have indeed artifactory, however the configuration is the following:

[global]
extra-index-url = https://address/artifactory/api/pypi/python-repo/simple
[install]
trusted-host = artifactory.address

I added:

[download]
trusted-host = artifactory.address

but the result didn't change.

I actually cannot find a wheel produced or stored anywhere for our packages.
Where are they coming from with the old download syntax?

I also noticed that I am on pip 8 which deprecates --download already, and
on latest documentation <https://pip.pypa.io/en/stable/user_guide/#id20>,
they still reference only that old syntax, but they also mention something

Note that pip install --download will look in your wheel cache first,
before trying to download from PyPI. If you've never installed your
requirements before, you won't have a wheel cache for those items.

In that case, since some of the packages never provided a whl, how would I
have them in my wheel cache?

A quick check of PyPI doesn't show a wheel for fastdtw 0.3.0. Have you
perhaps got an extra index configured somewhere? And if so, did you
only configure it for the "install" command, and not for the
"download" command?
On 19 January 2017 at 16:11, Diego Costantini

Post by Diego Costantini
Hi,
on pip version 8 (I know we are at 9 now :) ) I started getting

deprecation

Post by Diego Costantini
warnings when using "pip install --download".
So I did what suggested and moved to "pip download".
This change did not preserve the behavior.
Before I was getting all whl dependencies downloaded, now some of them

are

Post by Diego Costantini
tar.gz.
pip install --download=../lib "$SRC_PYTHON_PACKAGE"
Collecting fastdtw==0.3.0 (from myPackage)
Saved /path/fastdtw-0.3.0-cp27-none-any.whl
pip download -d ../lib "$SRC_PYTHON_PACKAGE"
Collecting fastdtw==0.3.0 (from myPackage)
Using cached fastdtw-0.3.0.tar.gz
Saved /path/fastdtw-0.3.0.tar.gz
Why did this happen and what are the implications of having a tar.gz vs

whl

Post by Diego Costantini
(I did not find anything conclusive in my search so far)?
1 - deps built by me as tar.gz (how did they end up in my folder as whl
before?)
2 - 3rd party deps
3 - Looking for the cache, I cannot find the tar.gz in my system
--
You received this message because you are subscribed to the Google

Groups

Post by Diego Costantini
"virtualenv" group.
To unsubscribe from this group and stop receiving emails from it, send

an
<javascript:>.

Post by Diego Costantini
Visit this group at https://groups.google.com/group/python-virtualenv.
For more options, visit https://groups.google.com/d/optout.

...

[Donald Stufft]

Note that pip install --download will look in your wheel cache first, before trying to download from PyPI. If you've never installed your requirements before, you won't have a wheel cache for those items.
In that case, since some of the packages never provided a whl, how would I have them in my wheel cache?

The wheel cache is filled with wheels that pip built locally. It’s possible that ``pip install —download`` gives you one of those locally built wheels but pip download does not.

—
Donald Stufft

[Diego Costantini]

Ok, if I remove the whl from the cache, it gives me the tar.gz, and now
this is expected.
And as we said perhaps download does not check for cached wheels anymore
(where can I ask to confirm this? the github project sends here, perhaps I
create a new topic with that exact question).

How is that whl ending up there?
Here is my lead: I am using pybuilder and calling
install_runtime_dependencies against a requirements.txt, and this caches
the whl.
Maybe this is a feature of pybuilder to speed up future builds, but in this
case I am back asking what's in the topic title: what are the implications
of having whl vs tar.gz? (consider that I am shipping those dependencies
along in my deployments, which are not guaranteed 100% to be the same
combination of SW/HW of the building agent)

Post by Diego Costantini
I also noticed that I am on pip 8 which deprecates --download already, and
on latest documentation <https://pip.pypa.io/en/stable/user_guide/#id20>,
they still reference only that old syntax, but they also mention something

Note that pip install --download will look in your wheel cache first,
before trying to download from PyPI. If you've never installed your
requirements before, you won't have a wheel cache for those items.

In that case, since some of the packages never provided a whl, how would I
have them in my wheel cache?
The wheel cache is filled with wheels that pip built locally. It’s
possible that ``pip install —download`` gives you one of those locally
built wheels but pip download does not.
—
Donald Stufft
--
You received this message because you are subscribed to a topic in the
Google Groups "virtualenv" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/python-virtualenv/1trXi2y8C-Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/python-virtualenv.
For more options, visit https://groups.google.com/d/optout.

...