miserable python crap

[ggodw000]

needed to generate executable in linux in python, which sort of requies to install package called bbfreeze, those python packages, confusing formats/installers (i.e. wheel, egg, sdisk, pip, setuptool) are seem to be in a miserable crappy state as ever, amateuristic!

http://docs.python-guide.org/en/latest/ ... r-code-ref

Code: Select all

[root@localhost python.lab]# pip install bbfreeze
Collecting bbfreeze
  Using cached bbfreeze-1.1.3-py27-none-any.whl
Collecting bbfreeze-loader<1.2.0,>=1.1.0 (from bbfreeze)
  Using cached bbfreeze-loader-1.1.0.zip
Collecting altgraph==0.9 (from bbfreeze)
Building wheels for collected packages: bbfreeze-loader
  Running setup.py bdist_wheel for bbfreeze-loader ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-yNw3ym/bbfreeze-loader/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmppzeht1pip-wheel- --python-tag cp27:

  ------ bbfreeze-loader 1.1.0 configuration ------
  PYTHONVERSION = python2.7
  bits = 64
  darwin = False
  linker = gcc -Wl,-z,relro -Xlinker -export-dynamic
  platform = Linux-3.10.0-327.el7.x86_64-x86_64-with-redhat-7.2-Maipo
  static_library =
  symbolic_functions_bug = False
  sys.executable = /usr/bin/python
  sys.maxunicode = 0x10ffff
  sys.version = 2.7.5 (default, Oct 11 2015, 17:47:16)
  [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]
  unix = True
  win32 = False
  -------------------------------------------------

  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/_bbfreeze_loader
  copying _bbfreeze_loader/__init__.py -> build/lib.linux-x86_64-2.7/_bbfreeze_loader
  running build_ext
  building '_bbfreeze_loader/console' extension
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/_bbfreeze_loader
  gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _bbfreeze_loader/console.c -o build/temp.linux-x86_64-2.7/_bbfreeze_loader/console.o
  In file included from _bbfreeze_loader/console.c:2:0:
  _bbfreeze_loader/loader_impl.h:3:20: fatal error: Python.h: No such file or directory
   #include <Python.h>
                      ^
  compilation terminated.
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for bbfreeze-loader
  Running setup.py clean for bbfreeze-loader
Failed to build bbfreeze-loader
Installing collected packages: bbfreeze-loader, altgraph, bbfreeze
  Running setup.py install for bbfreeze-loader ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-yNw3ym/bbfreeze-loader/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-KUxi7k-record/install-record.txt --single-version-externally-managed --compile:

    ------ bbfreeze-loader 1.1.0 configuration ------
    PYTHONVERSION = python2.7
    bits = 64
    darwin = False
    linker = gcc -Wl,-z,relro -Xlinker -export-dynamic
    platform = Linux-3.10.0-327.el7.x86_64-x86_64-with-redhat-7.2-Maipo
    static_library =
    symbolic_functions_bug = False
    sys.executable = /usr/bin/python
    sys.maxunicode = 0x10ffff
    sys.version = 2.7.5 (default, Oct 11 2015, 17:47:16)
    [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]
    unix = True
    win32 = False
    -------------------------------------------------

    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/_bbfreeze_loader
    copying _bbfreeze_loader/__init__.py -> build/lib.linux-x86_64-2.7/_bbfreeze_loader
    running build_ext
    building '_bbfreeze_loader/console' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/_bbfreeze_loader
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _bbfreeze_loader/console.c -o build/temp.linux-x86_64-2.7/_bbfreeze_loader/console.o
    In file included from _bbfreeze_loader/console.c:2:0:
    _bbfreeze_loader/loader_impl.h:3:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-yNw3ym/bbfreeze-loader/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-KUxi7k-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-yNw3ym/bbfreeze-loader/

[simeonz]

Apparently, you need to install python-dev. I am not sure if it is even the package's fault. May be the pip system is not capable enough to track and resolve such dependencies.

[ggodw000]

if you were my employee i'd blast you into pieces and show you the exit door for saying "it is not package's" fault or more appropriately you have produced something like this and defend it. However, that is a pipe dream i am imagining, I will explain as humanly as possible why it is fucking wrong degenerate piece of crap this is:
- just looking at the errors, no one with the sane mind can figure out, python-dev is needed. (just search for python-dev in this mess of logs and you will not find it).
- this will serve as a derivative for all sort of hell, considering thousands or millions of developers will attempt to install and run into same issue and running around like a headless chiecken.
- obviously dependency rule is whatever applicable for situations like this just did not work. Possibly a result for amateur development...

[simeonz]

if you were my employee i'd blast you into pieces and show you the exit door for saying "it is not package's" fault or more appropriately you have produced something like this and defend it. However, that is a pipe dream i am imagining,

That was close.

However, that is a pipe dream i am imagining, I will explain as humanly as possible why it is fucking wrong degenerate piece of crap this is:
- just looking at the errors, no one with the sane mind can figure out, python-dev is needed. (just search for python-dev in this mess of logs and you will not find it).

Looking at the logs, I can immediately tell you that the C api headers are missing. This is the interface that enables writing python modules in C. Such development resources would be packaged separately from the main python package. So, I just google the name of the package. Ironically, I immediately find an answer to almost the same problem on StackOverflow. You do realize that this is abnormally terminating installation process, and you need to do some troubleshooting if you want to work around the issue. But since you are ... an employer, you probably don't have such habits.

- this will serve as a derivative for all sort of hell, considering thousands or millions of developers will attempt to install and run into same issue and running around like a headless chiecken.

You are installing currently unmaintained package (check the page on PyPi) that supports unconventional use of a language. How many millions of developers do you think are trying this exactly? Besides there are other ways to do this, including by greping the strace output of the python process, etc.

obviously dependency rule is whatever applicable for situations like this just did not work. Possibly a result for amateur development...

You did not understand my previous comment at all. But I am optimistic.

[ggodw000]

i am very sad i am talking to some amateruist degenerate like you. I don't care about all these technical craps you are talking about. You probably went through muds and shits to gain the knowledge that telling other to do through mud. But let me try to have you understand (likes of you never understand but I will give it a try).
Packages are designed to be usable for other consumers (in this case developers that are not knowledgeable about your package and they want to use it and expect it to work). Think about black box, of which inside you care nothing about. Input is what is required from consumers, in this case the developer who is intending to use this package. Output is the what you get from using the package. The instruction that were published should be accurate and sole source of information that should give users what is needed to use the package in this case unfortunately it failed miserably in that respect. You can design some crap like this and defend it anyway but you nevertheless failed in delivering a quality and robustness that create value and therefore save time. If you spend 100 hours developing package and expecting your consumers to spend another 100 hours debugging your package, looking to see what is wrong and another 100 hours on learning curve to get to the same level of knowledge as you, then yours has no value, they are better doing their own.

More disgusting is the fact, that someone like you telling me producing a half-hearted shitcrap like this and then telling me (a consumer) that I need to do a troublehooting and fool around internet for answers. In a strict typical business environment, that is waste of resource, time and eventually a money being wasted. Missed information is that may or may not be anywhere in a junkyard like internet is a business risk. By not properly documenting and producing sound and robust package, one has incurred the siginificant business risk as the add'l development steals away from precious company resource. Typically a word comes out of some degenerate who does not own what they do, which unfortunately you seem to have similar mentality.

But I will end my lecture here, no matter what ever you put here, i ain't gonna wrestle around, i got better things to do in life.

[ggodw000]

i should not have used harsh language like this here and toward you, but was quite frustrated with this crap. Anyways i recommended to management that our software policy to not depend on python's open source public packages due to persistent stability issues and abolish any software project that uses public packages (they are already having constant build and runtime problem for months) and stick strictly core python packages to save on company effort, time and resource. But my point remains the same, I dont normally tolerate the half-hearted product and look for excellence or close it, not all sorts of excuses.

[StudlyCaps]

I had a post all ready to go, but since you seem to have realized you overreacted to simeonz, there's not much point submitting it.

If your company chooses to use free software so you can save on developer man-hours that's their decision. However, the developers/maintainers don't owe you anything, not support, not updates, not bug-fixes. If you want those thing then you pay for the tools you use, or make your own. It really is that simple.
Free software doesn't exist to cut corporate overhead, if you find something you can use then good for you, but I'm sure whoever wrote bbfreeze doesn't care.

[ggodw000]

I had a post all ready to go, but since you seem to have realized you overreacted to simeonz, there's not much point submitting it.

If your company chooses to use free software so you can save on developer man-hours that's their decision. However, the developers/maintainers don't owe you anything, not support, not updates, not bug-fixes. If you want those thing then you pay for the tools you use, or make your own. It really is that simple.
Free software doesn't exist to cut corporate overhead, if you find something you can use then good for you, but I'm sure whoever wrote bbfreeze doesn't care.

Yes, time is more expensive than money, most open source project end up costing more because of the mistaken belief that it is saving them $$, and developers of OS project thinking they are doing favor. So in a strict business sense, I'd prefer to pay for closed source that are known to guarantee certain levels of quality at the bottom line. I already hinted about severe example some department going through who chose to constantly upgrade to latest python packages for months doing enormous amount of waste. The exceptions are truly great open source project that are known for stability but I'd in no change vouch for any third party python packages.

Code: Select all

Free software doesn't exist to cut corporate overhead, if you find something you can use then good for you, but I'm sure whoever wrote bbfreeze doesn't care.

And final point, you already mentioned, how the certain software package can be good or decent if the developer who did it does not care?

[StudlyCaps]

I totally agree, even though there is some excellent free software out there, there is also a whole lot that isn't suitable for a commercial project. Generally speaking though, if something is important to your business you'd be better off paying for it.

Most free software is targetted at hobbyists though, and for a hobbyist it's ok, I mean if you're choosing to work on something with no practical value, for fun, then the time you take to do it really has no monetary value.

Edit: Re: the dev doesn't care:
I meant more in your individual case, the dev has no motivation to make their package more useful to you. They made something for their own reasons, whatever those might be, and then they said "hey, if this is useful for you, take it." From the look, bbfreeze is no longer maintained, so obviously they didn't feel like using any more of their free time to improve it.

[ggodw000]

I still disagree, the way of thinking that you are doing particular software for hobby does not means you should skimp on quality, it is eventually a "shot at your own leg" thing. I have several pet projects in C, python and shell scripts, I have tried to promote to fellow workers and associate but no avail, but that is not the point. Many of them serve to automate certain purpose, but I also realize the burden being incurred in myself as I spend more time improving it.

Why? at some point, you are going to become your own customer of your code you wrote and will be a victim of it, this is specially true if you did something and havent touched it for let's say 6 months or more. Or after 2 years, you look back and decided to improve your code then you certainly not going to remember details of how your super-complicated function consisting of 4000 lines code doing critical stuff, now you fell on your trap! You have to re-study your own work, huge investment!

Do not tell me you would not mind that, I have done it and hated doing it or worse losing it thinking the stuff I did on my own is lesser importance or not serious enough etc.,. Now every project I do, be it a personal or corporate, I make sure documented it well and kept it in a git source version control. Sound practice and yes they are valuable!

[StudlyCaps]

That's fair. I don't think I necessarily disagree, I think it's always a good idea to make things to the best of your ability and I don't like the OSS attitude of "if you don't like it, just fork the source". I also know how much frustration is causes!
That said, "perfect is the enemy of good". I'd rather someone published a working tool with a dodgy installer than just mothball it.
If I had some code which took a week to write but the installer was broken and it takes someone an hour to install then I'm still saving them 39 hours by making it available. In the ideal world, I fix the installer, but time is in short supply for everyone and sometimes that won't happen, would you rather the package was just never released at all?

[Schol-R-LEA]

A few points:

First, while I love some aspects of the Python language, I do have to agree that as a development platform, it is as shaky as the San Andreas Fault. I would agree that it is amateurish in many places, especially in regards to things like package deployment. It is better than most, open-source or closed-source, but it still sucks and is something of a mess.

However, to paraphrase The Brain paraphrasing Orson Welles (NSFW), if you can find a jury that would state that there is anything in the IT field that isn't pure amateur hour, I will make cheese for you. No one in this field even knows how to build professional-quality software yet, and it won't be until well after your grand-children's grand-children's time that anyone will. Any claims to either professionalism or engineering standards at this stage are laughable.

We, as a field, are at the equivalent level of Neolithic villagers building mud huts, and you want to build an aluminum cantilever bridge - a very small one, yes, but still one which requires knowledge no one has yet. If Python's package system sucks, it is because every package system today sucks - indeed every program to date sucks, and that will continue to be true for the foreseeable future.

More to the point, FOSS is amateur by definition. You shouldn't be using FOSS software at all unless you are willing to commit the time needed to adapt it, and can afford to work within an open license framework for the parts that are FOSS. Otherwise, don't use FOSS.

And no, I don't care one whit about the rhetoric of open-source versus closed-source. None of that really applies once commercial interests are involved in the software's production and distribution - because the point of FOSS isn't that it isn't distributed on an exclusively commercial basis, it's that the source code should be available to be inspected by outside parties to ensure that the engineering of the code (such as we can manage with our mud huts) is up to par. Unfortunately, as you've seen, that theory conflicts with the actual practice of software development by amateurs, where usually only those parts which are 'interesting' get proper effort.

This isn't really the point, though. The point is that while it is possible to use FOSS tools and even FOSS libraries in commercial projects, it takes a lot of effort to make it work without breaking either the code or the licenses, and for such projects a closed-source solution is often better - even if it is technically inferior to the FOSS solution, because the technical issues pale in significance to the legal, cultural and interpersonal ones.

Actually, interpersonal communication issues overshadow everything else in any non-trivial software project, but that's another topic altogether.

In any case, there is a closed-source compatible implementation of Python from ActiveState called ActivePython, which I understand fixes a lot of these issues - but only for their own packages and those they support.

In other words, whoever decided to use a FOSS toolchain didn't do their due diligence.

Second, there is the question of why the project requires bbfreeze, or more specifically, why it requires a binary executable. This is not at all standard practice in the Linux world, or anywhere else for that matter, when working with Python. Let me be blunt - if you need a binary executable, you shouldn't be using Python. It is simply the wrong tool for the job, because the 'executable' you will end up with will just be a wrapper around the Python interpreter anyway, plus the bytecode-compiled Python program - it will bloat your project with code you don't actually need, and do nothing to prevent RE that just using a .pyc wouldn't.

Turning the .pyc file into a data section of an ELF file? Not a great plan - and as far as I can tell, that's exactly what tools like bbfreeze, cx_freeze, and (on Windows) Py2Exe do.

While it is possible to compile Python - or any other language - into an executable, in the case of a language which has and uses live code interpretation (exec() and the like) the executable will need to include the interpreter, at least if the program uses it. As I understand it, Python programs use it a lot internally, even if it isn't used explicitly, meaning that the interpreter would still end up bundled into any compiled package.

It looks as if this whole issue of building an executable is meant to support closed source software distribution, which means you shouldn't be using the CPython toolchain (the most widely used FOSS implementation, and presumably the one you are working in) at all - you'd be distributing the interpreter inside a closed source package, which is a non-no. If you can't distribute source, don't use a language that requires you to distribute source for part of the package.

In other words, whoever decided to use Python in particular didn't do their due diligence.

Third, @ggodw000 never mentioned which version of Python the project was using. This is relevant because if it is in Python 3.x, the package won't work no matter what. Despite the insistence of others to the contrary, Python 2.x and Python 3.x (and similarly, Ruby 1.x and Ruby 2.x) are better treated as separate languages, rather than versions of the same language - going from 2.x to 3.x entailed almost as many changes as going from, say, Pascal to Modula-2. Again, this is stated very explicitly in the documentation on PyPi.

So, if it is a Python 3.x project using a tool that doesn't work in Python 3.x, that's a due diligence problem again.

Finally, @ggodw000 mentioned the various and admittedly contradictory Python packaging systems, which seems to really be the crux of this matter. There are two problems here, however. First, the whole Python ecosystem is sort of caught in the middle of a very slow moving shift between two standards, Egg and Wheel. Egg is much older, the design of it is a bit sloppy, and the format is not very well standardized, while Wheel is much better designed and documented but the implementation isn't really ready for prime time. Only about a third of PyPi projects have moved to Wheel, to date.

While the primary distribution tool for CPython, pip, supports both, they don't play nice together a lot of the time much as RPM, Portage, and .deb files don't. It is usually easier to convert an Egg to a Wheel yourself rather than trying to get them to work together. This is something that most Python documentation covering them mentions.

This is something that you should have been aware of before committing to using Python, in the process of determining how to conduct the project. Should I repeat what I said earlier?

So tell me again: who is it that is being unprofessional here?

[Korona]

I'm not going to comment on Python issues as I have nothing meaningful to say about that topic. However:

More to the point, FOSS is amateur by definition.

This statement is just plain wrong. There are many FOSS projects that are maintained by professional programmers that get paid for this job. Linux (Red Hat et al), GCC (same), LLVM (Apple, Google), Chromium (Google) and Firefox (Mozilla), just to name a few. Almost every core component of a typical Linux desktop or server is sponsored and maintained by some big company. FOSS != every shitty patch gets committed to master without review by a professional programmer. Also FOSS != the maintainers do not get paid by software companies.

[Schol-R-LEA]

This is all true; however, given that my primary assertion was that - regardless of salary and other aspects - there is no such thing as professional software development, and there won't be any in our lifetimes, because we are groping blindly around the elephant no matter what our pay scales or credentials are, I don't really see it as terribly relevant

[dchapiesky]

I was just going say that I hate python too.....

but also say that the package distribution systems python initially used were groundbreaking at the time and contributed to its popularity and pervasiveness.... kind of like how gopher opened up whole new vistas of the internet when people were still impressed by rsh....

that the package distribution systems have failed to evolve to deal with today's issues is, in my opinion, attributed to the sheer weight of pythons popularity and entrenched pervasiveness...