recentpopularlog in

kme : python   1336

« earlier  
coding style - Single quotes vs. double quotes in Python - Stack Overflow
I like to use double quotes around strings that are used for interpolation or that are natural language messages, and single quotes for small symbol-like strings, but will break the rules if the strings contain quotes, or if I forget. I use triple double quotes for docstrings and raw string literals for regular expressions even if they aren't needed.
python  quotes  quotingstyle  codingstyle  bestpractices 
6 days ago by kme
psf/black: The uncompromising Python code formatter
Black is a well-behaved Unix-style command-line tool:

* it does nothing if no sources are passed to it;
* it will read from standard input and write to standard output if - is used as the filename;
* it only outputs messages to users on standard error; exits with code 0 unless an internal error occurred (or --check was used).
python  codingstyle  codestyle  formatter 
6 days ago by kme
mchaput / whoosh / wiki / Home — Bitbucket
Whoosh is a fast, featureful full-text indexing and searching library implemented in pure Python. Programmers can use it to easily add search functionality to their applications and websites. Every part of how Whoosh works can be extended or replaced to meet your needs exactly.
python  fts  fulltextindex  searchandindex  purepython  library 
6 weeks ago by kme
Update: pdf-parser.py Version 0.7.0 | Didier Stevens
<code class="language-bash">
./pdf-parser.py -n document.pdf
./pdf-parser.py -s objstm document.pdf

# decode stream objects ('-f' = filter)
./pdf-parser.py -s objstm -f document.pdf

# force 'pdfid.py' to parse the output of above (even though it's
# missing a proper PDF header)
./pdf-parser.py -s objstm -f document.pdf | ./pdfid.py -n -f

# which is (I think?) is roughly the same as
./pdf-parser.py -a -O document.pdf
</code>
pdf  parser  reversing  reverseengineering  forensic  malware  analysis  commandline  python  video  streamobject  solution 
8 weeks ago by kme
Decompress FlateDecode Objects in PDF
Tips on how to get working in Python 3 here: https://stackoverflow.com/a/53609642/785213

<code class="language-python">import re
import zlib

pdf = open("some_doc.pdf", "rb").read()
stream = re.compile(r'.*?FlateDecode.*?stream(.*?)endstream', re.S)

for s in stream.findall(pdf):
s = s.strip('\r\n')
try:
print(zlib.decompress(s))
print("")
except:
pass</code>
python  pdf  reversing  forensics  objectstream  flatedecode  zlib 
8 weeks ago by kme
joe is broken · Issue #99 · karan/joe
is solved the update issue by going into the .joe-data directory and manually checking out the master branch. After checking out the branch, joe recognizes the .gitignore files.
joe  python  gitignore  bug  errormessage  needshelp 
october 2019 by kme
PEP 440 -- Version Identification and Dependency Specification | Python.org
For a given release identifier V.N, the compatible release clause is approximately equivalent to the pair of comparison clauses:

>= V.N, == V.*

This operator MUST NOT be used with a single segment version number such as ~=1.

For example, the following groups of version clauses are equivalent:

~= 2.2
>= 2.2, == 2.*

~= 1.4.5
>= 1.4.5, == 1.4.*
python  pip  semver  packaging  versionpinning  pinning  versioning  dependency  reference  dammitbrain 
october 2019 by kme
Getting Started With setuptools and setup.py — an_example_pypi_project v0.0.5 documentation
<code class="language-python">import os
from setuptools import setup

# Utility function to read the README file.
# Used for the long_description. It's nice, because now 1) we have a top level
# README file and 2) it's easier to type in the README file than to put a raw
# string in below ...
def read(fname):
return open(os.path.join(os.path.dirname(__file__), fname)).read()

setup(
name = "an_example_pypi_project",
version = "0.0.4",
author = "Andrew Carter",
author_email = "andrewjcarter@gmail.com",
description = ("An demonstration of how to create, document, and publish "
"to the cheese shop a5 pypi.org."),
license = "BSD",
keywords = "example documentation tutorial",
url = "http://packages.python.org/an_example_pypi_project",
packages=['an_example_pypi_project', 'tests'],
long_description=read('README'),
classifiers=[
"Development Status :: 3 - Alpha",
"Topic :: Utilities",
"License :: OSI Approved :: BSD License",
],
)</code>
python  install  setup.py  setuptools  packaging  template  dammitbrain 
october 2019 by kme
python - How can I simulate input to stdin for pyunit? - Stack Overflow
As of Python 3.5 "If you are patching builtins in a module then you don’t need to pass create=True, it will be added by default."

<code class="language-python">
@patch.object(module_under_test, "raw_input", create=True)
def test_using_decorator(self, raw_input):
raw_input.return_value = input_data = "123"
expected = int(input_data)

actual = module_under_test.function()

self.assertEqual(expected, actual)</code>
python  mock  monkeypatching  testing  unittest  solution 
october 2019 by kme
Evidlo/redrum: Reddit wallpaper changer. Does math to rank images.
<code class="language-ini">
## search these subreddits
subreddits = winterporn
earthporn
natureporn
spaceporn
jungleporn
astrophotography
californiapics
breathless
amateurphotography
amateurearthporn
eyecandy
bridgeporn
cabinporn
churchporn
megalithporn
summerporn
architectureporn
</code>
python  reddit  wallpaper  downloader  eyecandy  samplecode 
october 2019 by kme
Scrapy - xpath return parent node with content based on regex match - Stack Overflow
Not the solution I was looking for, but has a useful example of how to lower-case a text string (good idea for sorting case-insensitively).

<code class="language-python">def parse(self, response):
for href in response.xpath('//a[contains(translate(@href,"ABCDEFGHIJKLMNOPQRSTUVWXYZ","abcdefghijklmnopqrstuvwxyz"),"keyword")]/@href'):
full_url = response.urljoin(href.extract())
yield { 'url': full_url, }</code>

Note that XPath 2.0 seems to have 'upper-case' and 'lower-case' functions which simplify this process. Also note that (as of this writing, 2019-10-23) XmlStarlet does not support this function.
xml  xsl  xpath  python  scrapy  examplecode  textprocessing 
october 2019 by kme
undefined symbol: PyUnicodeUCS4_FromUnicode - Python
I ran into this with a wheel for python-levenshtein that was built for UCS-4, but my homebrew Python was UCS-2. Using 'pip install python-levenshtein --no-binary :all:' to ignore the wheel was the solution for me (source: https://stackoverflow.com/a/38263887/785213)
python  pip  wheel  unicode  sortof  solution 
september 2019 by kme
sqlite - Compile Python 3.4 with sqlite3 - Stack Overflow
There is also the option of pre-linking your custom Python build with your own-built sqlite3. (I had the same issue: the custom python was using the system-provided sqlite3, completely ignoring the sqlite3 I built).

Prefix your configure and make commands with:
<code class="language-bash">LD_RUN_PATH=$HOME/opt/lib configure LDFLAGS="-L$HOME/opt/lib" CPPFLAGS="-I$HOME/opt/include" …
LD_RUN_PATH=$HOME/opt/lib make</code>

so that the built python3 by default is linked to your sqlite3. This worked for me.
python  fromsource  sqlite  errormessage  linkererrors  solution 
september 2019 by kme
charles leifer | Compiling SQLite for use with Python Applications
<code class="language-python">>>> import sqlite3
>>> sqlite3.sqlite_version
'3.25.0'</code>
python  sqlite  fromsource  reference 
september 2019 by kme
python - How does the dependency resolution works with pip when a package specify a range and another doesn't? - Stack Overflow
I think I just understood "global dependency" resolution. IPython and ptpython require prompt_toolkit >= 2.something, but 'mycli' depends on something like 0.16. So installing 'mycli' with pip will always break IPython, and vice-versa.
python  dependencies  pip  dependencyhell  dependencyresolution  epiphany 
august 2019 by kme
windows - Run Python scripts in PowerShell directly - Super User
Edit the PATHEXT environment variable and add the .py extension.

Just add this line to your powershell profile:
<code class="language-powershell">$env:PATHEXT += ";.py"</code>

or you could just edit PATHEXT globally using the system settings (just start menu search for "Edit Environment variables for your account").


See https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_profiles; the MS TechNet article linked in the approved answer is looooong dead.
python  powershell  windows  cli  commandline  pathext  path  environmentvariable  shellscripting  solution  essential  movein  configuration 
august 2019 by kme
Issue 31652: make install fails: no module _ctypes - Python tracker
Had to build and install libffi into the PREFIX for Python-3.7.4 when building it from source on CentOS 7.

See also: https://bugs.python.org/issue34036
centos  centos7  fromsource  build  toolchain  compiler  errormessage  python  workaround  solution 
august 2019 by kme
Issue 34036: ModuleNotFoundError: No module named '_ctypes' when install Python 3.7 on Linux - Python tracker
Ran into this problem building Python 3.7.4 on the HPC cluster.
Ctypes is meant to be (at least it used to be) an optional module. If you don’t actually care about building ctypes, this might be the same as Issue 31652. Using “configure --without-ensurepip” was suggested as a workaround.
python  build  fromsource  compiler  errormessage  maybesolution 
august 2019 by kme
peritus/bumpversion: Version-bump your software with a single command
Pretty handily replaces what I used to use a complicated custom Makefile for, except that it *only* does version numbers, not "last updated" dates or anything like that.

Handy note:
Options on the command line take precedence over those from the config file, which take precedence over those derived from the environment and then from the defaults.
python  versionnumber  distribution  deployment  automation  alternativeto  makefile 
august 2019 by kme
Where do I find Python? - Apple Community
Asking questions is hard. Coming up with sympathetic answers in the face of profound misunderstandings is even harder.
python  apple  newbie  questions 
august 2019 by kme
python - What is the global default timeout - Stack Overflow | https://stackoverflow.com/
The answer is it's compiled into the kernel. Just use the 'timeout=' parameter to urlllib.request.urlopen if you're tired of waiting.
python  webdevel  timeout  urllib  sortof  solution 
july 2019 by kme
Issue 23195: Sorting with locale (strxfrm) does not work properly with Python3 on BSD or OS X - Python tracker
What is 'ln_LA' anyway?
The initial difference appears to be a long-standing BSD (including OS X) versus GNU/Linux platform difference. See, for example:
http://www.postgresql.org/message-id/18C8A481-33A6-4483-8C24-B8CE70DB7F27@eggerapps.at

Why there is no difference between en and fr UTF-8 is obvious when you look under the covers at the system locale definitions. This is on FreeBSD 10, OS X 10.10 is the same:

$ cd /usr/share/locale/fr_FR.UTF-8/
$ ls -l
total 8
lrwxr-xr-x 1 root wheel 28 Jan 16 2014 LC_COLLATE -> ../la_LN.US-ASCII/LC_COLLATE
lrwxr-xr-x 1 root wheel 17 Jan 16 2014 LC_CTYPE -> ../UTF-8/LC_CTYPE
lrwxr-xr-x 1 root wheel 30 Jan 16 2014 LC_MESSAGES -> ../fr_FR.ISO8859-1/LC_MESSAGES
-r--r--r-- 1 root wheel 36 Jan 16 2014 LC_MONETARY
lrwxr-xr-x 1 root wheel 29 Jan 16 2014 LC_NUMERIC -> ../fr_FR.ISO8859-1/LC_NUMERIC
-r--r--r-- 1 root wheel 364 Jan 16 2014 LC_TIME

For some reason US-ASCII is used for UTF-8 collation; this is also true for en_US.UTF-8 and de_DE.UTF-8, the only other ones I checked.

The postresq discussion and some earlier Python issues suggest using ICU to properly implement Unicode functions like collation across all platforms. But that has never been implemented in Python. Nosing Marc-Andre.
python  sorting  locale  collation  strings  macos  elcapitan  brokenness 
july 2019 by kme
Sorting strings properly is stupidly hard – Daniel Lemire's blog
However, I tried to test out the sorting on fr_ca locale and got the incorrect answer, which I found out was due to incorrect locale settings on Max OS X/BSD. On my machine, fr_FR.UTF-8 collation is linked to la_LN.US-ASCII
sorting  ishard  collation  strings  devel  javascript  python  pitfalls  macos  elcapitan  brokenness 
july 2019 by kme
python - UnicodeEncodeError: 'ascii' codec can't encode character u'xa0' in position 20: ordinal not in range(128) - Stack Overflow | https://stackoverflow.com/
So, even if your format string is Unicode, you *still* need to encode it?

Weird.

<code class="language-python">
print(u"{}\u00a0{}\u00a0{}\u00a0".format('non', 'breaking', 'spaces').encode.('utf-8'))
</code>
This is a classic python unicode pain point! Consider the following:
<code>
a = u'bats\u00E0'
print a
=> batsà
</code>

All good so far, but if we call str(a), let's see what happens:
<code>
str(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe0' in position 4: ordinal not in range(128)
</code>

Oh dip, that's not gonna do anyone any good! To fix the error, encode the bytes explicitly with .encode and tell python what codec to use:
<code>
a.encode('utf-8')
=> 'bats\xc3\xa0'
print a.encode('utf-8')
=> batsà
</code>
python  unicode  stringconcatenation  encoding  utf8  solution  reference 
july 2019 by kme
python - Is it possible to open a locked sqlite database in read only mode? - Stack Overflow
SQLITE_OPEN_READONLY flag - look into this. Probably only part of the C API, which Python's sqlite3 won't necessarily support.

This is the solution I ended up using:
You could try just copying the database file (e.g., with the system utility cp) and using that snapshot for reading purposes
python  sqlite  concurrency  database  locking  errormessage  maybesolution 
june 2019 by kme
Python Example
But... what's the deal with this web site?

Update: Ryan made it - https://github.com/ryanlr
python  samplecode 
june 2019 by kme
Python SQLite: database is locked - Stack Overflow
Got this message when trying to use my 'ffhist' Python script while Firefox was open.
python  sqlite  dba  database  locking  errormessage 
june 2019 by kme
« earlier      
per page:    204080120160

Copy this bookmark:





to read