recentpopularlog in

kme : butwhy   63

What the heck are you actually using NoSQL for? - High Scalability -
Has an impressive list of references at the bottom, for further reading.
API limiting. This is a great fit for Redis as a rate limiting check needs to be made for every single API hit, which involves both reading and writing short-lived data.
webdevel  devel  database  dba  architecture  nosql  explained  butwhy  forethereferences 
10 weeks ago by kme
The #! magic, details about the shebang/hash-bang mechanism |
what's special about #!

#! was a great hack to make scripts look and feel like real executable binaries.

But, as a little summary, what's special about #!? (list mostly courtesy of David Korn)

- the interpretername must not contain blanks
- the length of the #! is much smaller than the maximum path length
- $PATH is not searched for the interpreter
- (apart from an absolute path, the #! line also accepts a relative path,
- and #!interpreter is equivalent to #!./interpreter,
- however, it's not of any practical use)
- the interpreter usually must no be a #! script again
- the handling of arguments in the #! line itself is varying
- the setuid mechanism may or may not be available for the script
- there's no way to express #!$SHELL

And why shebang? In music, '#' means sharp. So just shorten #! to sharp-bang. Or it might be derived from "shell bang". All this probably under the influence of the american slang idiom "the whole shebang" (everything, the works, everything involved in what is under consideration). See also the wiktionary, jargon dictionary or Merriam-Websters. Sometimes it's also called hash-bang, pound-bang, sha-bang/shabang, hash-exclam, or hash-pling (british, isn't it?).

According to Dennis M. Ritchie (email answer to Alex North-Keys) it seems it had no name originally.
And Doug McIllroy mentioned (TUHS mailing list), that the slang for # at Bell Labs most probably was "sharp" at the time.
bourne  bash  posix  shell  shebang  unix  linux  shellscripting  history  butwhy  explained 
february 2019 by kme
The Unix Shell: Instructor Notes |
Many people have questioned whether we should still teach the shell. After all, anyone who wants to rename several thousand data files can easily do so interactively in the Python interpreter, and anyone who’s doing serious data analysis is probably going to do most of their work inside the IPython Notebook or R Studio. So why teach the shell?

The first answer is, “Because so much else depends on it.” Installing software, configuring your default editor, and controlling remote machines frequently assume a basic familiarity with the shell, and with related ideas like standard input and output. Many tools also use its terminology (for example, the %ls and %cd magic commands in IPython).

The second answer is, “Because it’s an easy way to introduce some fundamental ideas about how to use computers.” As we teach people how to use the Unix shell, we teach them that they should get the computer to repeat things (via tab completion, ! followed by a command number, and for loops) rather than repeating things themselves. We also teach them to take things they’ve discovered they do frequently and save them for later re-use (via shell scripts), to give things sensible names, and to write a little bit of documentation (like comment at the top of shell scripts) to make their future selves’ lives better.

The third answer is, “Because it enables use of many domain-specific tools and compute resources researchers cannot access otherwise.” Familiarity with the shell is very useful for remote accessing machines, using high-performance computing infrastructure, and running new specialist tools in many disciplines. We do not teach HPC or domain-specific skills here but lay the groundwork for further development of these skills. In particular, understanding the syntax of commands, flags, and help systems is useful for domain specific tools and understanding the file system (and how to navigate it) is useful for remote access.

Finally, and perhaps most importantly, teaching people the shell lets us teach them to think about programming in terms of function composition. In the case of the shell, this takes the form of pipelines rather than nested function calls, but the core idea of “small pieces, loosely joined” is the same.

Installing Bash and a reasonable set of Unix commands on Windows always involves some fiddling and frustration. Please see the latest set of installation guidelines for advice, and try it out yourself before teaching a class.

Tab completion sounds like a small thing: it isn’t. Re-running old commands using !123 or !wc isn’t a small thing either, and neither are wildcard expansion and for loops. Each one is an opportunity to repeat one of the big ideas of Software Carpentry: if the computer can repeat it, some programmer somewhere will almost certainly have built some way for the computer to repeat it.

Building up a pipeline with four or five stages, then putting it in a shell script for re-use and calling that script inside a for loop, is a great opportunity to show how “seven plus or minus two” connects to programming. Once we have figured out how to do something moderately complicated, we make it re-usable and give it a name so that it only takes up one slot in working memory rather than several. It is also a good opportunity to talk about exploratory programming: rather than designing a program up front, we can do a few useful things and then retroactively decide which are worth encapsulating for future re-use.
shellscripting  unix  linux  shell  bash  butwhy  programming  sevenplusorminustwo  teaching  workshop  reference  advice  bestpractices 
february 2019 by kme
bash - Set and Shopt - Why Two? - Unix & Linux Stack Exchange |
The difference is in the changed environment variable used by bash. Setting with the set command results in $SHELLOPTS. Setting with the shopt command results in $BASHOPTS.
bourne  bash  shell  configfile  configuration  history  butwhy  explained 
february 2019 by kme
how am i supposed to remember ipv6 addresses? - Open Forum | DSLReports Forums |
Short answer: local name servers or /etc/hosts. Or copy-paste.
On a broader note, I am always amazed at the comment 'How can I remember' when dealing with computers. Excuse me but isn't that what computers are for? Is the concept of making a file and then copy/paste foreign to most people? I don't have to remember anything not that I could remember my 32 digit login's and passwords, as an example. As for IPV6, for the vast majority of home users, it's irrelevant.
ipv6  ip  internet  homenetworking  networking  sysadmin  butwhy  answered 
january 2019 by kme
networking - Is there any benefit to using IPv6 on my home network? - Super User |
Just about sums it up:
But with IPv4 you can't give all of your kitchen appliances billions of IP addresses!
ipv6  homenetworking  networking  butwhy 
january 2019 by kme
Apple iPad Pro review 2018: the fastest iPad is still an iPad - The Verge |
Apple seems to want it both ways with the iPad Pro: it loves to tout the iPad’s laptop-dwarfing sales figures and industry-leading performance, but when pushed on the iPad’s limitations, the company insists that the iPad is still an ongoing attempt to build the future of computing, not a laptop replacement.

But after eight years, this double-sided argument is no longer tenable. Unlike virtually every other computer, the iPad is a product of Apple’s singular vision: the company designs the display, the processor, the operating system, and the limits of the applications and accessories that plug into it. And after all this time, it’s clear that whatever roadblocks and frustrations exist in using the iPad Pro are there because Apple wants them there. There just aren’t that many excuses left.
ipad  apple  hardware  butwhy  photoshop 
december 2018 by kme
How original is Miller? |
No, really, why one more command-line data-manipulation tool? I wrote Miller because I was frustrated with tools like grep, sed, and so on being line-aware without being format-aware. The single most poignant example I can think of is seeing people grep data lines out of their CSV files and sadly losing their header lines. While some lighter-than-SQL processing is very nice to have, at core I wanted the format-awareness of RecordStream combined with the raw speed of the Unix toolkit. Miller does precisely that.
butwhy  unix  commandline  textprocessing  onethingwell  philosophy  explained 
december 2018 by kme
DORMANDO - / /usr /home split
I swear I first read this in a Slackware install guide, but I had the habit of making /, /usr, /home, as well as /tmp or /var split into different mounts. The theory was that the files in / changed least, /usr more often, with /home, /tmp, and /var touched most often.

So in the case of power loss, kernel panic, or cords-wrapped-around-chairs, the filesystems were split in highest odds of being able to boot again to rescue the system. / contained just enough to boot, /usr had more useful utilities, and /home had all your stuff. I personally benefitted from this split several times on servers and a home machine.
unix  filesystem  sysadmin  butwhy  explained 
march 2018 by kme
ParsingLs - Greg's Wiki |
Best explanation came from the BashPitfalls article:
Never try to parse the output of ls. ls is just plain unnecessary. It's an external command whose output is intended specifically to be read by a human, not parsed by a script.
bash  unix  shellscripting  filemanagement  bestpractices  newbie  reference  butwhy 
november 2017 by kme
issue tracking - JIRA: to close or to resolve? - Stack Overflow |
Typical issue workflow is the person working on the bug resolves it, and the person who opened the bug is the one who decides if the resolution is acceptable. If it is, they close it. If not, they re-open the bug for further discussion/work/wrangling.
bitbucket  jira  issuetracking  devel  question  butwhy  solution 
october 2017 by kme
Why isn’t there a switch or case statement in Python? - Design and History FAQ — Python 2.7.14 documentation |
You can do this easily enough with a sequence of if... elif... elif... else. There have been some proposals for switch statement syntax, but there is no consensus (yet) on whether and how to do range tests. See PEP 275 for complete details and the current status.

For cases where you need to choose from a very large number of possibilities, you can create a dictionary mapping case values to functions to call. For example:

def function_1(...):

functions = {'a': function_1,
'b': function_2,
'c': self.method_1, ...}

func = functions[value]

For calling methods on objects, you can simplify yet further by using the getattr() built-in to retrieve methods with a particular name:

def visit_a(self, ...):

def dispatch(self, value):
method_name = 'visit_' + str(value)
method = getattr(self, method_name)

It’s suggested that you use a prefix for the method names, such as visit_ in this example. Without such a prefix, if values are coming from an untrusted source, an attacker would be able to call any method on your object.
python  language  syntax  newbie  faq  butwhy 
october 2017 by kme
Python join: why is it string.join(list) instead of list.join(string)? - Stack Overflow
From the comments:
For easy memory and understanding, '"string".join(list)' declares that you are joining a list and converting to a string. It's result oriented. – Yumi Tada Dec 4 '17 at 12:26

It's because any iterable can be joined, not just lists, but the result and the "joiner" are always strings.


import urllib2
print '\n############\n'.join(
python  strings  lists  syntax  butwhy  solution 
october 2017 by kme
Why do humans get "goosebumps" when they are cold, or under other circumstances? - Scientific American
The contraction also causes the hair to stand up whenever the body feels cold. In animals with a thick hair coat this rising of hair expands the layer of air that serves as insulation. The thicker the hair layer, the more heat is retained. In people this reaction is useless because we do not have a hair coat, but goosebumps persist nevertheless.
butwhy  humans  homosapiens  evolution 
april 2017 by kme
TOML vs. JSON - Tom Limoncelli's EverythingSysadmin Blog
The other difference is around comments. One camp permits them and another camp doesn't. In operations often we need to be able to temporarily comment out a few lines, or include ad hoc messages. Operations people communicate by leaving breadcrumbs and todo items in files. Rather than commenting out some lines I could delete them and use version control to bring them back, but that is much more work. Also, often I write code in comments for the future. For example, as part of preparation for a recent upgrade, we added the future configuration lines to a file but commented them out. By including them, they could be proofread by coworkers.
toml  json  configuration  operations  sysadmin  breadcrumbs  documentation  butwhy  thisvsthat 
april 2017 by kme
Why does Cargo use toml? - The Rust Programming Language Forum
The TOML readme has a small section comparing itself to JSON, YAML and INI:

In some ways TOML is very similar to JSON: simple, well-specified, and maps easily to ubiquitous data types. JSON is great for serializing data that will mostly be read and written by computer programs. Where TOML differs from JSON is its emphasis on being easy for humans to read and write. Comments are a good example: they serve no purpose when data is being sent from one program to another, but are very helpful in a configuration file that may be edited by hand.

The YAML format is oriented towards configuration files just like TOML. For many purposes, however, YAML is an overly complex solution. TOML aims for simplicity, a goal which is not apparent in the YAML specification:

The INI format is also frequently used for configuration files. The format is not standardized, however, and usually does not handle more than one or two levels of nesting.


While I can not speak for the Rust authors, having used the different formats in my own projects (as well as JSON variants such as HCL) I believe they made the right choice.

I don't think they are. Especially security issues aren't emphasized enough. The biggest issue is that YAML attempts to be many things to many people. It's JSON, it's a serialization format, it's human readable, etc. I've been reading the mailing list for YAML for quite some time now.

To me YAML has become F-35 of the ML formats. That is to say overly complex, does many things and isn't really good at any particular thing. I mean, can YAML parser beat JSON parser at parsing JSON (I don't think it is possible, simply because YAML has more states)?
butwhy  rust  cargo  json  configfile  inifile 
april 2017 by kme
Why won't Apple make a touch screen laptop and iMac in combination with also keeping the trackpad, mouse, and keyboard? - Quora
Several of us retrofit third party touch screen interfaces onto a number of iMacs. The experience wasn't very satisfying, unless you set the software driver to do a combined "move here + click". Even then, you ended up with finger grease all over the thing, which significantly detracted from its utility for GUI and design work, and it was really annoying when pointing things out in terminal windows, and so on. Overall, it was not that useful, and only two of the 5 of us who had hacked up their iMacs left the touch screen enabled.

Again, this was the same reason given by the UX people.

The only surprising thing was the lack of a Bluetooth keyboard with a touchpad that was always in the same location relative to the spacebar, to give a laptop-like experience. I saw several people attempting to use third party versions of these things, with little luck. The problem came down to the ability to use the touchpad area without relocating your fingers away from their default locations for touch typing. Something which is easy with a keyboard the size of a small laptop keyboard, but much more difficult with a full size keyboard, even if you omit the numeric keypad in the design.

And yet again, the same reason for not supporting it that was given by the UX people.

Don't think that these things have not been tried out internally, or using UX labs and having people actually use prototype equipment that has the features; they almost certainly have for everything you could think might be a good idea to add to an iMac or MacBook.
apple  hardware  macbook  touchscreen  butwhy 
march 2017 by kme
Why don't MacBooks have two screens, one being a touch screen on the back not just for you but also for strangers? - Quora
There are various reasons. Here's some:

The lid would have to be thicker and Jony would throw a fit.
It would look silly and Jony would throw a second fit.
It would weigh more and Jony would throw a third fit.
It would cost more and Tim Cook would throw a fit.
It would be feature creep and checklist design, at which point Steve would rise up from beyond and smite whichever engineer and designer was responsible for such a travesty.
You are already carrying around a small touch screen in the iPhone - why replicate it attached to your laptop.
It would burn battery.
It would only be useful to you when the laptop was out of its case yet closed. A tiny amount of the time.
apple  hardware  touchscreen  butwhy 
march 2017 by kme
Environment variables for java installation - Stack Overflow
Set the following user environment variables (== environment variables of type user variables)

JAVA_HOME : C:\Program Files\Java\jdk1.8.0_25
PATH : your-unique-entries;%JAVA_HOME%\bin (make sure that the longish your-unique-entries does not contain any other references to another Java installation folder.

Notice that all these environment variables are derived from the "root" environment variable JAVA_HOME. This makes it easy to update your environment variables when updating the JDK. Just point JAVA_HOME to the fresh installation.
java  wtf  environmentvariables  windows  devel  maybesolution  butwhy 
march 2016 by kme
git Archives: Re: Does GIT has vc keywords like CVS/Subversion?
On Mon, 9 Oct 2006, Liu Yubao wrote:
> IMHO, I don't think keyword substitution is a good idea, as it will confuse
> the external diff/merge tools.

There are other reasons why it's a _horrible_ idea, like the fact that it
can mess up binary files etc (so if you do keyword substitution, you also
need to suddenly care _deeply_ whether a file is binary or not).

The whole notion of keyword substitution is just totally idiotic. It's
trivial to do "outside" of the actual content tracking, if you want to
have it when doing release trees as tar-balls etc.

- inside of the SCM, keyword substitution is pointless, since you have
much better tools available (like "git log filename")
- outside of the SCM, keyword substitution can make sense, but doing it
should be in helper scripts or something that can easily tailor it for
the actual need of that particular project.

For example, we actually do a certain kind of keyword subtituion for the
kernel. Look at the -git snapshots: the script that generates the snapshot
diffs has a simple sequence in it to "keyword substitute" the Makefile for
the EXTRAVERSION flag, so the diff will result in the Makefile having the
knowledge of which git SHA1 version the resulting patch was, even though
the thing isn't a git tree any more:

git-read-tree $CURCOMM
git-checkout-index Makefile
git-diff-index -m -p $RELTREE | gzip -9 > $STAGE/patch-$CURNAME.gz

So this is how to do keyword substitution in a _sane_ way.

Sure, we could do something like this as a git script, and support it
"natively", but the fact is, keyword substitution is just stupid.

git  keywordsubstitution  thereason  fromthehorsesmouth  butwhy 
march 2016 by kme
Learning Subversion: the mystery of .svn | Python Conquers The Universe
So now we know what a Subversion administrative directory contains.

The .svn admin directory contains pristine (unchanged) copies of files that were downloaded from the repository. (It contains a few other things, too.)
svn  versioncontrol  newbie  butwhy 
may 2015 by kme

Copy this bookmark:

to read