I don’t do a huge amount of Python, but when I do, it usually involves Scrapy.
As a result of not doing a lot of Python, it usually means I don’t have all the relevant gubbins (yes, that is the correct technical description) installed to make things ‘just work ™’.
On the contrary, things usually go quite wrong indeed:
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-I6du0W/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-hwGRh6-record/install-record.txt --single-version-externally-managed --compile --user failed with error code 1 in /tmp/pip-build-I6du0W/lxml
or
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-dLB7U_/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-KNAq8K-record/install-record.txt --single-version-externally-managed --compile --user failed with error code 1 in /tmp/pip-build-dLB7U_/lxml
Storing debug log for failure in /home/chris/.pip/pip.log
and so on.
Now, also, I am not the world’s foremost expert on Linux, and / or the package dependencies required to get Python to play nicely.
This is a polite way of saying I blindly follow the top answers on StackOverflow until whatever it is I am trying to install (e.g. Scrapy) installs without some horrendous error message. Hey, a little brutal honesty never hurt anybody.
So, to save my future sanity, here are all the things I ran today to get Scrapy 1.1 installed on Ubuntu 15.10:
➜ ~ python --version Python 2.7.10 ➜ ~ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 15.10 Release: 15.10 Codename: wily
You will need pip installed. (sudo apt-get install python-pip worked for me.)
Then this – I am aware there are duplicates, this is a concat of about 6 different attempts. Feel free to de-dupe if you want, it makes no difference (afaik):
sudo apt-get install build-essential autoconf libtool pkg-config python-opengl python-imaging python-pyrex python-pyside.qtopengl idle-python2.7 qt4-dev-tools qt4-designer libqtgui4 libqtcore4 libqt4-xml libqt4-test libqt4-script libqt4-network libqt4-dbus python-qt4 python-qt4-gl libgle3 python-dev libxml2-dev libxslt1-dev python-dev libxml2-dev libxslt1-dev python-dev libxml2-dev libxslt-dev python-dev lib32z1-dev build-essential libssl-dev libffi-dev python-dev
And then Scrapy installed ‘first time’ using:
pip install Scrapy
Marvellous.
If you’d like to learn more Python, I’m not the best person to speak to. I would highly recommend you check out one or more of the courses listed at CourseDuck.
Still not good 🙁
apt-get problems:
Note, selecting ‘libxslt1-dev’ instead of ‘libxslt-dev’
E: Unable to locate package lib32z1-dev
pip install scrapy problems:
collect2: error: ld returned 1 exit status
error: command ‘i686-linux-gnu-gcc’ failed with exit status 1
—————————————-
Command “/usr/bin/python -u -c “import setuptools, tokenize;__file__=’/tmp/pip-build-TY9l_h/lxml/setup.py’;f=getattr(tokenize, ‘open’, open)(__file__);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, __file__, ‘exec’))” install –record /tmp/pip-P8mryP-record/install-record.txt –single-version-externally-managed –compile” failed with error code 1 in /tmp/pip-build-TY9l_h/lxml/
I’m unsure how best to advise on the specific issue you are facing.
What I would do now – if needing to use Scrapy – would be to either use an Ansible role (google for “ansible scrapy” to find plenty), or maybe use a Docker container.
One tip would be to google for “ansible scrapy” and then even if not using Ansible, you could still follow the playbook(s) manually – e.g. https://github.com/arbabnazar/ansible-roles/blob/master/scrapy/tasks/main.yml – and see if they have any dependencies or steps you haven’t tried.
Sorry I can’t help further, but best of luck getting it working.