nhaliday : qra   283

 « earlier
Is there a common method for detecting the convergence of the Gibbs sampler and the expectation-maximization algorithm? - Quora
In practice and theory it is much easier to diagnose convergence in EM (vanilla or variational) than in any MCMC algorithm (including Gibbs sampling).

https://www.quora.com/How-can-you-determine-if-your-Gibbs-sampler-has-converged
There is a special case when you can actually obtain the stationary distribution, and be sure that you did! If your markov chain consists of a discrete state space, then take the first time that a state repeats in your chain: if you randomly sample an element between the repeating states (but only including one of the endpoints) you will have a sample from your true distribution.

One can achieve this 'exact MCMC sampling' more generally by using the coupling from the past algorithm (Coupling from the past).

Otherwise, there is no rigorous statistical test for convergence. It may be possible to obtain a theoretical bound for the convergence rates: but these are quite difficult to obtain, and quite often too large to be of practical use. For example, even for the simple case of using the Metropolis algorithm for sampling from a two-dimensional uniform distribution, the best convergence rate upper bound achieved, by Persi Diaconis, was something with an astronomical constant factor like 10^300.

In fact, it is fair to say that for most high dimensional problems, we have really no idea whether Gibbs sampling ever comes close to converging, but the best we can do is use some simple diagnostics to detect the most obvious failures.
nibble  q-n-a  qra  acm  stats  probability  limits  convergence  distribution  sampling  markov  monte-carlo  ML-MAP-E  checking  equilibrium  stylized-facts  gelman  levers  mixing  empirical  plots  manifolds  multi  fixed-point  iteration-recursion  heuristic  expert-experience  theory-practice  project
october 2019 by nhaliday
How can lazy importing be implemented in Python? - Quora
The Mercurial revision control system has the most solid lazy import implementation I know of. Note well that it's licensed under the GPL, so you can't simply use that code in a project of your own.
- Bryan O'Sullivan
q-n-a  qra  programming  python  howto  examples  performance  tricks  time  latency-throughput  yak-shaving  expert-experience  hg  build-packaging  oss  property-rights  intellectual-property
august 2019 by nhaliday
Could diving into water save you from a hail of bullets like in movies? - Quora
I believe that while water could help keep you safe in a hail from bullets, there are many flaws, such as being stuck in a low depth pool or running out of oxygen.
q-n-a  qra  embodied  safety  death  fighting  stylized-facts  trivia  fluid  swimming  survival  arms  martial  prepping  short-circuit
august 2019 by nhaliday
Information Processing: Beijing 2019 Notes
Trump, the trade war, and US-China relations came up frequently in discussion. Chinese opinion tends to focus on the long term. Our driver for a day trip to the Great Wall was an older man from the countryside, who has lived only 3 years in Beijing. I was surprised to hear him expressing a very balanced opinion about the situation. He understood Trump's position remarkably well -- China has done very well trading with the US, and owes much of its technological and scientific development to the West. A recalibration is in order, and it is natural for Trump to negotiate in the interest of US workers.

China's economy is less and less export-dependent, and domestic drivers of growth seem easy to identify. For example, there is still a lot of low-hanging fruit in the form of "catch up growth" -- but now this means not just catching up with the outside developed world, but Tier 2 and Tier 3 cities catching up with Tier 1 cities like Beijing, Shanghai, Shenzhen, etc.

China watchers have noted the rapidly increasing government and private sector debt necessary to drive growth here. Perhaps this portends a future crisis. However, I didn't get any sense of impending doom for the Chinese economy. To be fair there was very little inkling of what would happen to the US economy in 2007-8. Some of the people I met with are highly placed with special knowledge -- they are among the most likely to be aware of problems. Overall I had the impression of normalcy and quiet confidence, but perhaps this would have been different in an export/manufacturing hub like Shenzhen. [ Update: Today after posting this I did hear something about economic concerns... So situation is unclear. ]

Innovation is everywhere here. Perhaps the most obvious is the high level of convenience from the use of e-payment and delivery services. You can pay for everything using your mobile (increasingly, using just your face!), and you can have food and other items (think Amazon on steroids) delivered quickly to your apartment. Even museum admissions can be handled via QR code.

A highly placed technologist told me that in fields like AI or computer science, Chinese researchers and engineers have access to in-depth local discussions of important arXiv papers -- think StackOverflow in Mandarin. Since most researchers here can read English, they have access both to Western advances, and a Chinese language reservoir of knowledge and analysis. He anticipates that eventually the pace and depth of engineering implementation here will be unequaled.

IVF and genetic testing are huge businesses in China. Perhaps I'll comment more on this in the future. New technologies, in genomics as in other areas, tend to be received more positively here than in the US and Europe.

...

Note Added: In the comments AG points to a Quora post by a user called Janus Dongye Qimeng, an AI researcher in Cambridge UK, who seems to be a real China expert. I found these posts to be very interesting.

Infrastructure development in poor regions of China

Size of Chinese internet social network platforms

Can the US derail China 2025? (Core technology stacks in and outside China)

Huawei smartphone technology stack and impact of US entity list interdiction (software and hardware!)

Agriculture at Massive Scale

US-China AI competition

More recommenations: Bruno Maçães is one of my favorite modern geopolitical thinkers. A Straussian of sorts (PhD under Harvey Mansfield at Harvard), he was Secretary of State for European Affairs in Portugal, and has thought deeply about the future of Eurasia and of US-China relations. He spent the last year in Beijing and I was eager to meet with him while here. His recent essay Equilibrium Americanum appeared in the Berlin Policy Journal. Podcast interview -- we hope to have him on Manifold soon :-)
hsu  scitariat  china  asia  thucydides  tech  technology  ai  automation  machine-learning  trends  the-bones  links  reflection  qra  q-n-a  foreign-policy  world  usa  trade  nationalism-globalism  great-powers  economics  research  journos-pundits  straussian
july 2019 by nhaliday
What kind of problems give an MLE (memory limit exceeded)? - Quora
To be fair, memory limit exceeded is a pretty rare occurrence. Problems aren't usually set such that you have to optimize for memory.
q-n-a  qra  programming  oly  oly-programming  space-complexity  stylized-facts  computer-memory  prioritizing
july 2019 by nhaliday
Alon Amit's answer to Why is there no formal definition for a set in math? How can we make any statement about sets (and therefore all of math) if we don’t even know what it is? - Quora
In the realm of mathematics, an object is what it does (I keep quoting Tim Gowers with this phrase, and I will likely do so many more times). The only thing that matters about points, lines, real numbers, sets, functions, groups and tempered distributions is the properties and features and rules they obey. What they “are” is of no concern.

I've seen this idea in a lot of different places
q-n-a  qra  math  lens  abstraction  essence-existence  analytical-holistic  forms-instances  big-picture  aphorism  axioms  definition  characterization  zooming
july 2019 by nhaliday
Which of Haskell and OCaml is more practical? For example, in which aspect will each play a key role? - Quora
- Tikhon Jelvis,

Haskell.

This is a question I'm particularly well-placed to answer because I've spent quite a bit of time with both Haskell and OCaml, seeing both in the real world (including working at Jane Street for a bit). I've also seen the languages in academic settings and know many people at startups using both languages. This gives me a good perspective on both languages, with a fairly similar amount of experience in the two (admittedly biased towards Haskell).

And so, based on my own experience rather than the languages' reputations, I can confidently say it's Haskell.

Parallelism and Concurrency

...

Libraries

...

Typeclasses vs Modules

...

In some sense, OCaml modules are better behaved and founded on a sounder theory than Haskell typeclasses, which have some serious drawbacks. However, the fact that typeclasses can be reliably inferred whereas modules have to be explicitly used all the time more than makes up for this. Moreover, extensions to the typeclass system enable much of the power provided by OCaml modules.

...

Of course, OCaml has some advantages of its own as well. It has a performance profile that's much easier to predict. The module system is awesome and often missed in Haskell. Polymorphic variants can be very useful for neatly representing certain situations, and don't have an obvious Haskell analog.

While both languages have a reasonable C FFI, OCaml's seems a bit simpler. It's hard for me to say this with any certainty because I've only used the OCaml FFI myself, but it was quite easy to use—a hard bar for Haskell's to clear. One really nice use of modules in OCaml is to pass around values directly from C as abstract types, which can help avoid extra marshalling/unmarshalling; that seemed very nice in OCaml.

However, overall, I still think Haskell is the more practical choice. Apart from the reasoning above, I simply have my own observations: my Haskell code tends to be clearer, simpler and shorter than my OCaml code. I'm also more productive in Haskell. Part of this is certainly a matter of having more Haskell experience, but the delta is limited especially as I'm working at my third OCaml company. (Of course, the first two were just internships.)

Both Haskell and OCaml are uniquivocally superb options—miles ahead of any other languages I know. While I do prefer Haskell, I'd choose either one in a pinch.

--
I've looked at F# a bit, but it feels like it makes too many tradeoffs to be on .NET. You lose the module system, which is probably OCaml's best feature, in return for an unfortunate, nominally typed OOP layer.

I'm also not invested in .NET at all: if anything, I'd prefer to avoid it in favor of simplicity. I exclusively use Linux and, from the outside, Mono doesn't look as good as it could be. I'm also far more likely to interoperate with a C library than a .NET library.

If I had some additional reason to use .NET, I'd definitely go for F#, but right now I don't.

https://www.reddit.com/r/haskell/comments/3huexy/what_are_haskellers_critiques_of_f_and_ocaml/
https://www.reddit.com/r/haskell/comments/3huexy/what_are_haskellers_critiques_of_f_and_ocaml/cub5mmb/
Thinking about it now, it boils down to a single word: expressiveness. When I'm writing OCaml, I feel more constrained than when I'm writing Haskell. And that's important: unlike so many others, what first attracted me to Haskell was expressiveness, not safety. It's easier for me to write code that looks how I want it to look in Haskell. The upper bound on code quality is higher.

...

Perhaps it all boils down to OCaml and its community feeling more "worse is better" than Haskell, something I highly disfavor.

...

Laziness or, more strictly, non-strictness is big. A controversial start, perhaps, but I stand by it. Unlike some, I do not see non-strictness as a design mistake but as a leap in abstraction. Perhaps a leap before its time, but a leap nonetheless. Haskell lets me program without constantly keeping the code's order in my head. Sure, it's not perfect and sometimes performance issues jar the illusion, but they are the exception not the norm. Coming from imperative languages where order is omnipresent (I can't even imagine not thinking about execution order as I write an imperative program!) it's incredibly liberating, even accounting for the weird issues and jinks I'd never see in a strict language.

This is what I imagine life felt like with the first garbage collectors: they may have been slow and awkward, the abstraction might have leaked here and there, but, for all that, it was an incredible advance. You didn't have to constantly think about memory allocation any more. It took a lot of effort to get where we are now and garbage collectors still aren't perfect and don't fit everywhere, but it's hard to imagine the world without them. Non-strictness feels like it has the same potential, without anywhere near the work garbage collection saw put into it.

...

The other big thing that stands out are typeclasses. OCaml might catch up on this front with implicit modules or it might not (Scala implicits are, by many reports, awkward at best—ask Edward Kmett about it, not me) but, as it stands, not having them is a major shortcoming. Not having inference is a bigger deal than it seems: it makes all sorts of idioms we take for granted in Haskell awkward in OCaml which means that people simply don't use them. Haskell's typeclasses, for all their shortcomings (some of which I find rather annoying), are incredibly expressive.

In Haskell, it's trivial to create your own numeric type and operators work as expected. In OCaml, while you can write code that's polymorphic over numeric types, people simply don't. Why not? Because you'd have to explicitly convert your literals and because you'd have to explicitly open a module with your operators—good luck using multiple numeric types in a single block of code! This means that everyone uses the default types: (63/31-bit) ints and doubles. If that doesn't scream "worse is better", I don't know what does.

...

There's more. Haskell's effect management, brought up elsewhere in this thread, is a big boon. It makes changing things more comfortable and makes informal reasoning much easier. Haskell is the only language where I consistently leave code I visit better than I found it. Even if I hadn't worked on the project in years. My Haskell code has better longevity than my OCaml code, much less other languages.

http://blog.ezyang.com/2011/02/ocaml-gotchas/
One observation about purity and randomness: I think one of the things people frequently find annoying in Haskell is the fact that randomness involves mutation of state, and thus be wrapped in a monad. This makes building probabilistic data structures a little clunkier, since you can no longer expose pure interfaces. OCaml is not pure, and as such you can query the random number generator whenever you want.

However, I think Haskell may get the last laugh in certain circumstances. In particular, if you are using a random number generator in order to generate random test cases for your code, you need to be able to reproduce a particular set of random tests. Usually, this is done by providing a seed which you can then feed back to the testing script, for deterministic behavior. But because OCaml's random number generator manipulates global state, it's very easy to accidentally break determinism by asking for a random number for something unrelated. You can work around it by manually bracketing the global state, but explicitly handling the randomness state means providing determinism is much more natural.
q-n-a  qra  programming  pls  engineering  nitty-gritty  pragmatic  functional  haskell  ocaml-sml  dotnet  types  arrows  cost-benefit  tradeoffs  concurrency  libraries  performance  expert-experience  composition-decomposition  comparison  critique  multi  reddit  social  discussion  techtariat  reflection  review  random  data-structures  numerics  rand-approx  sublinear  syntax  volo-avolo  causation  scala  jvm  ecosystem  metal-to-virtual
june 2019 by nhaliday
LeetCode - The World's Leading Online Programming Learning Platform
very much targeted toward interview prep
https://www.quora.com/Is-LeetCode-Online-Judges-premium-membership-really-worth-it
This data is especially valuable because you get to know a company's interview style beforehand. For example, most questions that appeared in Facebook interviews have short solution typically not more than 30 lines of code. Their interview process focus on your ability to write clean, concise code. On the other hand, Google style interviews lean more on the analytical side and is algorithmic heavy, typically with multiple solutions to a question - each with a different run time complexity.
programming  tech  career  working-stiff  recruiting  interview-prep  algorithms  problem-solving  oly-programming  multi  q-n-a  qra  comparison  stylized-facts  facebook  google  cost-benefit  homo-hetero  startups  organization  alien-character  🖥  contest  puzzles  accretion  transitions  money-for-time
june 2019 by nhaliday
The End of the Editor Wars » Linux Magazine
Moreover, even if you assume a broad margin of error, the pollings aren't even close. With all the various text editors available today, Vi and Vim continue to be the choice of over a third of users, while Emacs well back in the pack, no longer a competitor for the most popular text editor.

https://www.quora.com/Are-there-more-Emacs-or-Vim-users
I believe Vim is actually more popular, but it's hard to find any real data on it. The best source I've seen is the annual StackOverflow developer survey where 15.2% of developers used Vim compared to a mere 3.2% for Emacs.

Oddly enough, the report noted that "Data scientists and machine learning developers are about 3 times more likely to use Emacs than any other type of developer," which is not necessarily what I would have expected.

[ed. NB: Vim still dominates overall.]

https://pinboard.in/u:nhaliday/b:6adc1b1ef4dc

Time To End The vi/Emacs Debate: https://cacm.acm.org/blogs/blog-cacm/226034-time-to-end-the-vi-emacs-debate/fulltext

Vim, Emacs and their forever war. Does it even matter any more?: https://blog.sourcerer.io/vim-emacs-and-their-forever-war-does-it-even-matter-any-more-697b1322d510
Like an episode of “Silicon Valley”, a discussion of Emacs vs. Vim used to have a polarizing effect that would guarantee a stimulating conversation, regardless of an engineer’s actual alignment. But nowadays, diehard Emacs and Vim users are getting much harder to find. Maybe I’m in the wrong orbit, but looking around today, I see that engineers are equally or even more likely to choose any one of a number of great (for any given definition of ‘great’) modern editors or IDEs such as Sublime Text, Visual Studio Code, Atom, IntelliJ (… or one of its siblings), Brackets, Visual Studio or Xcode, to name a few. It’s not surprising really — many top engineers weren’t even born when these editors were at version 1.0, and GUIs (for better or worse) hadn’t been invented.

...

… both forums have high traffic and up-to-the-minute comment and discussion threads. Some of the available statistics paint a reasonably healthy picture — Stackoverflow’s 2016 developer survey ranks Vim 4th out of 24 with 26.1% of respondents in the development environments category claiming to use it. Emacs came 15th with 5.2%. In combination, over 30% is, actually, quite impressive considering they’ve been around for several decades.

What’s odd, however, is that if you ask someone — say a random developer — to express a preference, the likelihood is that they will favor for one or the other even if they have used neither in anger. Maybe the meme has spread so widely that all responses are now predominantly ritualistic, and represent something more fundamental than peoples’ mere preference for an editor? There’s a rather obvious political hypothesis waiting to be made — that Emacs is the leftist, socialist, centralized state, while Vim represents the right and the free market, specialization and capitalism red in tooth and claw.

How is Emacs/Vim used in companies like Google, Facebook, or Quora? Are there any libraries or tools they share in public?: https://www.quora.com/How-is-Emacs-Vim-used-in-companies-like-Google-Facebook-or-Quora-Are-there-any-libraries-or-tools-they-share-in-public
In Google there's a fair amount of vim and emacs. I would say at least every other engineer uses one or another.

Among Software Engineers, emacs seems to be more popular, about 2:1. Among Site Reliability Engineers, vim is more popular, about 9:1.
--
People use both at Facebook, with (in my opinion) slightly better tooling for Emacs than Vim. We share a master.emacs and master.vimrc file, which contains the bare essentials (like syntactic highlighting for the Hack language). We also share a Ctags file that's updated nightly with a cron script.

Beyond the essentials, there's a group for Emacs users at Facebook that provides tips, tricks, and major-modes created by people at Facebook. That's where Adam Hupp first developed his excellent mural-mode (ahupp/mural), which does for Ctags what iDo did for file finding and buffer switching.
--
For emacs, it was very informal at Google. There wasn't a huge community of Emacs users at Google, so there wasn't much more than a wiki and a couple language styles matching Google's style guides.

https://trends.google.com/trends/explore?date=all&geo=US&q=%2Fm%2F07zh7,%2Fm%2F01yp0m

https://www.quora.com/Why-is-interest-in-Emacs-dropping
And it is still that. It’s just that emacs is no longer unique, and neither is Lisp.

Dynamically typed scripting languages with garbage collection are a dime a dozen now. Anybody in their right mind developing an extensible text editor today would just use python, ruby, lua, or JavaScript as the extension language and get all the power of Lisp combined with vibrant user communities and millions of lines of ready-made libraries that Stallman and Steele could only dream of in the 70s.

In fact, in many ways emacs and elisp have fallen behind: 40 years after Lambda, the Ultimate Imperative, elisp is still dynamically scoped, and it still doesn’t support multithreading — when I try to use dired to list the files on a slow NFS mount, the entire editor hangs just as thoroughly as it might have in the 1980s. And when I say “doesn’t support multithreading,” I don’t mean there is some other clever trick for continuing to do work while waiting on a system call, like asynchronous callbacks or something. There’s start-process which forks a whole new process, and that’s about it. It’s a concurrency model straight out of 1980s UNIX land.

But being essentially just a decent text editor has robbed emacs of much of its competitive advantage. In a world where every developer tool is scriptable with languages and libraries an order of magnitude more powerful than cranky old elisp, the reason to use emacs is not that it lets a programmer hit a button and evaluate the current expression interactively (which must have been absolutely amazing at one point in the past).

https://www.reddit.com/r/emacs/comments/bh5kk7/why_do_many_new_users_still_prefer_vim_over_emacs/

more general comparison, not just popularity:
Differences between Emacs and Vim: https://stackoverflow.com/questions/1430164/differences-between-Emacs-and-vim

https://www.reddit.com/r/emacs/comments/9hen7z/what_are_the_benefits_of_emacs_over_vim/

https://unix.stackexchange.com/questions/986/what-are-the-pros-and-cons-of-vim-and-emacs

https://www.quora.com/Why-is-Vim-the-programmers-favorite-editor
- Adrien Lucas Ecoffet,

Because it is hard to use. Really.

However, the second part of this sentence applies to just about every good editor out there: if you really learn Sublime Text, you will become super productive. If you really learn Emacs, you will become super productive. If you really learn Visual Studio… you get the idea.

Here’s the thing though, you never actually need to really learn your text editor… Unless you use vim.

...

For many people new to programming, this is the first time they have been a power user of… well, anything! And because they’ve been told how great Vim is, many of them will keep at it and actually become productive, not because Vim is particularly more productive than any other editor, but because it didn’t provide them with a way to not be productive.

They then go on to tell their friends how great Vim is, and their friends go on to become power users and tell their friends in turn, and so forth. All these people believe they became productive because they changed their text editor. Little do they realize that they became productive because their text editor changed them[1].

This is in no way a criticism of Vim. I myself was a beneficiary of such a phenomenon when I learned to type using the Dvorak layout: at that time, I believed that Dvorak would help you type faster. Now I realize the evidence is mixed and that Dvorak might not be much better than Qwerty. However, learning Dvorak forced me to develop good typing habits because I could no longer rely on looking at my keyboard (since I was still using a Qwerty physical keyboard), and this has made me a much more productive typist.

Technical Interview Performance by Editor/OS/Language: https://triplebyte.com/blog/technical-interview-performance-by-editor-os-language
[ed.: I'm guessing this is confounded to all hell.]

The #1 most common editor we see used in interviews is Sublime Text, with Vim close behind.

Emacs represents a fairly small market share today at just about a quarter the userbase of Vim in our interviews. This nicely matches the 4:1 ratio of Google Search Trends for the two editors.

...

Vim takes the prize here, but PyCharm and Emacs are close behind. We’ve found that users of these editors tend to pass our interview at an above-average rate.

On the other end of the spectrum is Eclipse: it appears that someone using either Vim or Emacs is more than twice as likely to pass our technical interview as an Eclipse user.

...

In this case, we find that the average Ruby, Swift, and C# users tend to be stronger, with Python and Javascript in the middle of the pack.

...

Here’s what happens after we select engineers to work with and send them to onsites:

[Python does best.]

There are no wild outliers here, but let’s look at the C++ segment. While C++ programmers have the most challenging time passing Triplebyte’s technical interview on average, the ones we choose to work with tend to have a relatively easier time getting offers at each onsite.

The Rise of Microsoft Visual Studio Code: https://triplebyte.com/blog/editor-report-the-rise-of-visual-studio-code
This chart shows the rates at which each editor's users pass our interview compared to the mean pass rate for all candidates. First, notice the preeminence of Emacs and Vim! Engineers who use these editors pass our interview at significantly higher rates than other engineers. And the effect size is not small. Emacs users pass our interview at a rate 50… [more]
news  linux  oss  tech  editors  devtools  tools  comparison  ranking  flux-stasis  trends  ubiquity  unix  increase-decrease  multi  q-n-a  qra  data  poll  stackex  sv  facebook  google  integration-extension  org:med  politics  stereotypes  coalitions  decentralized  left-wing  right-wing  chart  scale  time-series  distribution  top-n  list  discussion  ide  parsimony  intricacy  cost-benefit  tradeoffs  confounding  analysis  crosstab  pls  python  c(pp)  jvm  microsoft  golang  hmm  correlation  debate  critique  quora  contrarianism  ecosystem  DSL  techtariat  org:com  org:nat  cs
june 2019 by nhaliday
Should I go for TensorFlow or PyTorch?
Honestly, most experts that I know love Pytorch and detest TensorFlow. Karpathy and Justin from Stanford for example. You can see Karpthy's thoughts and I've asked Justin personally and the answer was sharp: PYTORCH!!! TF has lots of PR but its API and graph model are horrible and will waste lots of your research time.

--

...

Updated Mar 12
Update after 2019 TF summit:

TL/DR: previously I was in the pytorch camp but with TF 2.0 it’s clear that Google is really going to try to have parity or try to be better than Pytorch in all aspects where people voiced concerns (ease of use/debugging/dynamic graphs). They seem to be allocating more resources on development than Facebook so the longer term currently looks promising for Google. Prior to TF 2.0 I thought that Pytorch team had more momentum. One area where FB/Pytorch is still stronger is Google is a bit more closed and doesn’t seem to release reproducible cutting edge models such as AlphaGo whereas FAIR released OpenGo for instance. Generally you will end up running into models that are only implemented in one framework of the other so chances are you might end up learning both.
q-n-a  qra  comparison  software  recommendations  cost-benefit  tradeoffs  python  libraries  machine-learning  deep-learning  data-science  sci-comp  tools  google  facebook  tech  competition  best-practices  trends  debugging  expert-experience  ecosystem  theory-practice  pragmatic  wire-guided  static-dynamic  state  academia  frameworks  open-closed
may 2019 by nhaliday
How much would it cost to crawl 1 billion sites using rented AWS servers/bandwidth? - Quora
The best way IMHO to do such a crawl would be to recruit a group of say 100-1000 of your friends, and their friends, and write a simple distributed app running in background on their machines, when they sit idle or are lightly used. This way you will be amortizing their monthly broadband bills, with their monthly quotas (e.g. Comcast 250GB) largely unused anyway. I would think that you can get dozens of Mbps of cross bandwidth in such a network, which could do the job in a matter of months.

BTW, if you really meant 1 billion sites, as opposed to pages, multiply the above bills by 100x (average number of pages per site).

--

There is no need for you to crawl. Someone has already done the job for you. Common Crawl https://commoncrawl.org/ is a periodic crawl of the internet, and the results are stored in Amazon S3. You can directly use the results without any charge for any kink of analysis you want to do.
q-n-a  qra  quixotic  programming  engineering  search  minimum-viable  internet  web  huge-data-the-biggest  howto  init  advice  money  cost-benefit  strategy  scaling-tech  system-design  move-fast-(and-break-things)
may 2019 by nhaliday
Vladimir Novakovski's answer to What financial advice would you give to a 21-year-old? - Quora
Learn economics and see that investment and consumption levels (as percentages) depend only marginally on age and existing net worth and mostly on your risk preferences and utility function.
qra  q-n-a  oly  advice  reflection  personal-finance  ORFE  outcome-risk  investing  time-preference  age-generation  dependence-independence  economics
february 2019 by nhaliday
Does left-handedness occur more in certain ethnic groups than others?
Yes. There are some aboriginal tribes in Australia who have about 70% of their population being left-handed. It’s also more than 50% for some South American tribes.

The reason is the same in both cases: a recent past of extreme aggression with other tribes. Left-handedness is caused by recessive genes, but being left-handed is a boost when in hand-to-hand combat with a right-handed guy (who usually has trained extensively with other right-handed guys, as this disposition is genetically dominant so right-handed are majority in most human populations, so lacks experience with a left-handed). Should a particular tribe enter too much war time periods, it’s proportion of left-handeds will naturally rise. As their enemy tribe’s proportion of left-handed people is rising as well, there’s a point at which the natural advantage they get in fighting disipates and can only climb higher should they continuously find new groups to fight with, who are also majority right-handed.

...

So the natural question is: given their advantages in 1-on-1 combat, why doesn’t the percentage grow all the way up to 50% or slightly higher? Because there are COSTS associated with being left-handed, as apparently our neural network is pre-wired towards right-handedness - showing as a reduced life expectancy for lefties. So a mathematical model was proposed to explain their distribution among different societies

THE FIGHTING HYPOTHESIS: STABILITY OF POLYMORPHISM IN HUMAN HANDEDNESS

http://gepv.univ-lille1.fr/downl...

Further, it appears the average left-handedness for humans (~10%) hasn’t changed in thousands of years (judging by the paintings of hands on caves)

Frequency-dependent maintenance of left handedness in humans.

Handedness frequency over more than 10,000 years

[ed.: Compare with Julius Evola's "left-hand path".]
q-n-a  qra  trivia  cocktail  farmers-and-foragers  history  antiquity  race  demographics  bio  EEA  evolution  context  peace-violence  war  ecology  EGT  unintended-consequences  game-theory  equilibrium  anthropology  cultural-dynamics  sapiens  data  database  trends  cost-benefit  strategy  time-series  art  archaeology  measurement  oscillation  pro-rata  iteration-recursion  gender  male-variability  cliometrics  roots  explanation  explanans  correlation  causation  branches
july 2018 by nhaliday
Is the human brain analog or digital? - Quora
The brain is neither analog nor digital, but works using a signal processing paradigm that has some properties in common with both.

Unlike a digital computer, the brain does not use binary logic or binary addressable memory, and it does not perform binary arithmetic. Information in the brain is represented in terms of statistical approximations and estimations rather than exact values. The brain is also non-deterministic and cannot replay instruction sequences with error-free precision. So in all these ways, the brain is definitely not "digital".

At the same time, the signals sent around the brain are "either-or" states that are similar to binary. A neuron fires or it does not. These all-or-nothing pulses are the basic language of the brain. So in this sense, the brain is computing using something like binary signals. Instead of 1s and 0s, or "on" and "off", the brain uses "spike" or "no spike" (referring to the firing of a neuron).
q-n-a  qra  expert-experience  neuro  neuro-nitgrit  analogy  deep-learning  nature  discrete  smoothness  IEEE  bits  coding-theory  communication  trivia  bio  volo-avolo  causation  random  order-disorder  ems  models  methodology  abstraction  nitty-gritty  computation  physics  electromag  scale  coarse-fine
april 2018 by nhaliday
What is the difference between general and special relativity? - Quora
General Relativity is, quite simply, needed to explain gravity.

Special Relativity is the special case of GR, when the metric is flat — which means no gravity.

You need General Relativity when the metric gets all curvy, and when things start to experience gravitation.
nibble  q-n-a  qra  explanation  physics  relativity  synthesis  hi-order-bits  ground-up  gravity  summary  aphorism  differential  geometry
november 2017 by nhaliday
Best Topology Olympiad ***EVER*** - Affine Mess - Quora
Most people take courses in topology, algebraic topology, knot theory, differential topology and what have you without once doing anything with a finite topological space. There may have been some quirky questions about such spaces early on in a point-set topology course, but most of us come out of these courses thinking that finite topological spaces are either discrete or only useful as an exotic counterexample to some standard separation property. The mere idea of calculating the fundamental group for a 4-point space seems ludicrous.

Only it’s not. This is a genuine question, not a joke, and I find it both hilarious and super educational. DO IT!!
nibble  qra  announcement  math  geometry  topology  puzzles  rec-math  oly  links  math.AT  ground-up  finiteness  math.GN
october 2017 by nhaliday
What kills, current or voltage? - Quora
Its an oversimplification to say that voltage kills or current kills and the cause of much misunderstanding.

You cannot have one without the other, therefore one could claim that answering one or the other is correct.

However, it is the current THROUGH key parts of the body that can be lethal. BUT even a lot of Current in a wire won't harm you if the current is constrained to the wire..

Specifically in order for you to be electrocuted, the voltage must be high enough to drive a lethal amount of current through your body over coming body resistance, the voltage must be applied in the right places so that the current path is through your (usually) heart muscle, and it must be long enough duration to stop the heart muscle due to fibrillation.

another good answer:
As I write this note, I’m looking at the textbook Basic Engineering Circuit Analysis by Irwin and Nelms (ISBN 0-471-48728-7). On page 449 the authors reference the work of Dr. John G. Webster who suggests the body resistance values that I have taken the liberty to sketch into this poor character.
nibble  q-n-a  qra  physics  electromag  dirty-hands  embodied  safety  short-circuit  IEEE  data  objektbuch  death
september 2017 by nhaliday
Why was the Catholic Church so opposed to heliocentrism (for example, in the Renaissance)? Why did they not simply claim that God lived in the Sun, so we go around Him? - Quora
The main reason the Catholic Church opposed the teaching of heliocentrism as a fact was that it was contrary to the science of the time.

Amongst the modern myths about early science is the persistent idea that the opposition to heliocentrism was one of "science" versus "religion". According to this story, early modern astronomers like Copernicus and Galileo "proved" the earth went around the sun and the other scientists of the time agreed. But the Catholic Church clung to a literal interpretation of the Bible and rejected this idea purely out of a fanatical faith, insisting that the earth had to be the centre of the cosmos because man was the pinnacle of all creation. Pretty much everything in this popular story is wrong.
q-n-a  qra  history  medieval  europe  the-great-west-whale  science  the-trenches  discovery  giants  mediterranean  religion  christianity  protestant-catholic  theos  being-right  physics  mechanics  space  iron-age  the-classics  censorship
september 2017 by nhaliday
Flows With Friction
To see how the no-slip condition arises, and how the no-slip condition and the fluid viscosity lead to frictional stresses, we can examine the conditions at a solid surface on a molecular scale. When a fluid is stationary, its molecules are in a constant state of motion with a random velocity v. For a gas, v is equal to the speed of sound. When a fluid is in motion, there is superimposed on this random velocity a mean velocity V, sometimes called the bulk velocity, which is the velocity at which fluid from one place to another. At the interface between the fluid and the surface, there exists an attraction between the molecules or atoms that make up the fluid and those that make up the solid. This attractive force is strong enough to reduce the bulk velocity of the fluid to zero. So the bulk velocity of the fluid must change from whatever its value is far away from the wall to a value of zero at the wall (figure 7). This is called the no-slip condition.

http://www.engineeringarchives.com/les_fm_noslip.html
The fluid property responsible for the no-slip condition and the development of the boundary layer is viscosity.
https://www.quora.com/What-is-the-physics-behind-no-slip-condition-in-fluid-mechanics
https://www.reddit.com/r/AskEngineers/comments/348b1q/the_noslip_condition/
https://www.researchgate.net/post/Can_someone_explain_what_exactly_no_slip_condition_or_slip_condition_means_in_terms_of_momentum_transfer_of_the_molecules
https://en.wikipedia.org/wiki/Boundary_layer_thickness
http://www.fkm.utm.my/~ummi/SME1313/Chapter%201.pdf
org:junk  org:edu  physics  mechanics  h2o  identity  atoms  constraint-satisfaction  volo-avolo  flux-stasis  chemistry  stat-mech  nibble  multi  q-n-a  reddit  social  discussion  dirty-hands  pdf  slides  lectures  qra  fluid  local-global  explanation
september 2017 by nhaliday
What is the best way to parse command-line arguments with Python? - Quora
- Anders Kaseorg

Use the standard optparse library.

It’s important to uphold your users’ expectation that your utility will parse arguments in the same way as every other UNIX utility. If you roll your own parsing code, you’ll almost certainly break that expectation in obvious or subtle ways.

Although the documentation claims that optparse has been deprecated in favor of argparse, which supports more features like optional option arguments and configurable prefix characters, I can’t recommend argparse until it’s been fixed to parse required option arguments in the standard UNIX way. Currently, argparse uses an unexpected heuristic which may lead to subtle bugs in other scripts that call your program.

consider also click (which uses the optparse behavior)
q-n-a  qra  oly  best-practices  programming  terminal  unix  python  libraries  gotchas  howto  pls  yak-shaving  integration-extension  protocol-metadata
august 2017 by nhaliday
In a medieval European society, what percentage of people were farmers/peasants, how many were clergy, and how many were nobles? - Quora
Peasants- around 85–90%
Clergy 1%
Nobility (including knights) around 5–10%

As a side note nobilty could be as low as 1%. only frontier nations such as Castile ( Spain) and Poland would be in the 10% range.

This graph of Imperial Russia, (which was still a feudal autocracy in 1897 and had an almost identical class structure to a medieval kingdom) is very useful, just remove the working class and make them peasants!

lots of data on 1086 England (from Domesday Book): https://faculty.history.wisc.edu/sommerville/123/123%2013%20Society.htm
D&D advice mixed w/ historical grounding: http://www222.pair.com/sjohn/blueroom/demog.htm
http://www.lordsandladies.org/
https://www.reddit.com/r/history/comments/4jnc14/what_percentage_of_medieval_societies_were_nobles/
q-n-a  qra  history  medieval  europe  early-modern  pre-ww2  russia  social-structure  lived-experience  data  economics  labor  distribution  class  agriculture  anthropology  broad-econ  multi  org:junk  britain  org:edu  pro-rata  efficiency  population  civil-liberty  food  inequality  elite  vampire-squid  demographics  reddit  social  discussion  malthus  visualization  time-series  feudal
august 2017 by nhaliday
Trump wants a troop surge in Afghanistan — it makes little sense - Business Insider
http://abcnews.go.com/US/us-involved-afghanistan-difficult/story?id=49341264
http://www.rollingstone.com/politics/news/afghanistan-the-making-of-a-narco-state-20141204
https://www.washingtonpost.com/news/checkpoint/wp/2016/01/26/the-u-s-was-supposed-to-leave-afghanistan-by-2017-now-it-might-take-decades/
https://www.theatlantic.com/international/archive/2017/08/solution-afghanistan-withdrawal-iran-russia-pakistan-trump/537252/
https://www.wsj.com/articles/the-macarthur-model-for-afghanistan-1496269058
neo-colonialism, sensible
https://spottedtoad.wordpress.com/2017/09/11/tora-bora/
The U.S. has been a reasonably successful steward of world peace along some dimensions, no doubt, but we seem to be particularly bad at colonialism for reasons the Battle of Tora Bora perhaps highlights- once a government (or even loosely affiliated military group) is in theory our ally, under our tutelage and cooperating with our military machine, we seem to have no ability to view its actions or abilities objectively. Maybe the reason Britain was, all-in-all more successful as a colonial power despite never exerting the kind of world military dominance the U.S. has since World War II is that, as representatives of a class-based and explicitly hierarchical society, the Eton boys running things for Britain never felt tempted to the kinds of faux egalitarianism that often guides American colonial ventures astray. In his excellent if self-indulgent account of walking across Afghanistan immediately after the Taliban’s fall, The Places in Between, Rory Stewart (an Eton boy turned world traveler and, later, an Iraq War provincial administrator and Tory MP) describes the policy wonks eager to take the reins of the new Central Asian Switzerland in 2001: ...

Mystery deepens over Chinese forces in Afghanistan: https://www.ft.com/content/0c8a5a2a-f9b7-11e6-9516-2d969e0d3b65
https://www.quora.com/Whats-the-geopolitical-importance-of-Afghanistan
news  org:lite  org:biz  current-events  trump  politics  government  military  foreign-policy  geopolitics  war  MENA  terrorism  world  developing-world  history  mostly-modern  iraq-syria  questions  roots  impetus  multi  org:mag  drugs  org:rec  rhetoric  proposal  policy  realpolitik  usa  antidemos  expansionism  conquest-empire  paul-romer  org:anglo  china  asia  free-riding  chart  neocons  statesmen  leadership  strategy  density  q-n-a  qra  iran  russia  tribalism  kinship  comparison  britain  egalitarianism-hierarchy  realness  prudence  ratty  unaffiliated  stories  class
august 2017 by nhaliday
The Scholar's Stage: Everything is Worse in China
My time here has thus given me a rare vantage point to judge many of the claims made over the course of these campaigns. In few places is this sort of outside perspective more useful than when judging the claims of an American jeremiad. Jeremiading is a fine art. Its practitioners hail from lands both left and right, but my sympathies lie with the cultural traditionalists. You know the type. In America they find little but a shallow husk. For some it is the husk of a nation once great; for others it is the decaying remains of Western civilization itself. Few of these gloom-filled minds deny that wonders have marked their days on this earth. It is not that advances do not happen. It is just that each celebrated advance masks hundreds of more quiet destructions. These laments for worlds gone by are poignant; the best are truly beautiful. The best of the best, however, do not just lament. Every one of their portraits of the past is a depiction of a future—or more properly, a way of living worth devoting a future to.

I have read a few of these books in 2017. The best of these (both for its lyricism and for the demands it places on the intellect) is Anthony Esolen's newest book, Out of the Ashes: Rebuilding American Culture. This blog is not the place for a full review. I plan to write a proper review for it and a few of the other recently published books of this type for a less personal publication than the Scholar's Stage. Here I will just share one of my strongest reactions to the book—a thought that occurred again and again as I drifted through its pages. Esolen presents a swarm of maladies sickening American society, ranging from a generation of children suffocated by helicopter parenting to a massive state bureaucracy openly hostile to virtuous living. My reaction to each of his carefully drawn portraits was the same: this problem is even worse in China.

Are you worried about political correctness gone awry, weaponized by mediocrities to defame the worthy, suffocating truth, holding honest inquiry hostage through fear and terror? That problem is worse in China.

Do you lament the loss of beauty in public life? Its loss as a cherished ideal of not just art and oratory but in the building of homes, chapels, bridges, and buildings? Its disappearance in the comings-and-goings of everyday life? That problem is worse in China.

Do you detest a rich, secluded, and self-satisfied cultural elite that despises, distrusts, and derides the uneducated and unwashed masses not lucky enough to live in one of their chosen urban hubs? That problem is worse in China.

Are you sickened by crass materialism? Wealth chased, gained, and wasted for nothing more than vain display? Are you oppressed by the sight of children denied the joys of childhood, guided from one carefully structured resume-builder to the next by parents eternally hovering over their shoulders? Do you dread a hulking, bureaucratized leviathan, unaccountable to the people it serves, and so captured by special interests that even political leaders cannot control it? Are you worried by a despotic national government that plays king-maker in the economic sphere and crushes all opposition to its social programs into the dust? Do you fear a culture actively hostile to the free exercise of religion? Hostility that not only permeates through every layer of society, but is backed by the awesome power of the state?

These too are all worse in China.

Only on one item from Esolen's catalogue of decline can American society plausibly be described as more self-destructive than China's. China has not hopped headlong down the rabbit's hole of gender-bending. The Chinese have thus far proved impervious to this nonsense. But it would not be meet to conclude from this that Chinese society's treatment of sex is healthier than the West's.

https://gnxp.nofe.me/2017/07/25/on-the-precipice-of-the-kali-yuga/
interesting comments:
https://gnxp.nofe.me/2017/07/25/on-the-precipice-of-the-kali-yuga/comment-page-1/#comment-3091
https://gnxp.nofe.me/2017/07/25/on-the-precipice-of-the-kali-yuga/comment-page-1/#comment-3093
https://gnxp.nofe.me/2017/07/25/on-the-precipice-of-the-kali-yuga/comment-page-1/#comment-3109
https://gnxp.nofe.me/2017/07/25/on-the-precipice-of-the-kali-yuga/comment-page-1/#comment-3130
Re: authoritarianism and all that. I sometimes describe modern China as “slouching towards totalitarianism.” Bill Bishop descried it recently as a “leninist panopticon.”

(e.g. here http://cmp.hku.hk/2017/07/20/big-data-big-concerns/ here https://amp.ft.com/content/5ec7093c-6e06-11e7-b9c7-15af748b60d0 and here https://news.cgtn.com/news/3d676a4e3267444e/share_p.html# ).

But I think we need to dispense with some illusions. The elites of the CPC are unrelentingly hostile towards the West. They are King Goujian. They won’t be satisfied until China has displaced the United States as the world’s super power and they have the power to control the entire Chinese diaspora. (For those not familiar with the last bit see here http://www.smh.com.au/interactive/2017/chinas-operation-australia/soft-power.html and http://insidestory.org.au/beijings-guoqing-versus-australias-way-of-life ). On the long term will not tolerate an India or Japan that is not subservient, and they are not afraid to interfere with protected liberties in Western countries as long as Chinese-speakers are involved. For the most part they get away with it, as the censorship and intimidation they exercise in Western China-towns is all done in the Chinese language.

https://www.quora.com/Is-Chinese-history-taught-unbiasedly-in-China-Are-historical-figures-portrayed-as-heroes-villains/answer/Jamin-Chen-1
https://archive.is/XVRRC
While the book is primarily designed for overseas Chinese (hence it is bilingual), it is published by the Chinese government and is used in some Chinese schools in America to teach Chinese history. So presumably, students in China are taught something similar in their schools.

...

The Korean War is unabashedly called the “War to Resist US Aggression and Aid Korea,” while the Chinese title adds the phrase 保家卫国, or “defend our country and homes.” Notice how the book does not mention anything about the North Korean invasion, but it does mention how the US sent troops to Korea and how the Chinese involvement in the war “crushed the imperialists’ aggressive ambitions.”

In most US/Western textbooks, only three events in Chinese history post-1949 are extensively covered: the Great Leap Forward, Cultural Revolution, and Tiananmen Square, while other events are scantly acknowledged. This book covers all of Chinese history up until around 1999 (the year of Macau’s return to China), but between 1949 and 1999 it mentions three events: the “War to Resist US Aggression and Aid Korea,” Zhou Enlai’s diplomacy, Deng Xiaoping’s Reform and Opening Up, and Hong Kong’s return to China.

...
--
I actually agree that Western textbooks have a more objective view of history, or at least they’re better at hiding their bias.

My point is that there is bias in every country’s textbooks; how much bias is present and how the bias manifests is another question.
--
And you did a good job. I’ve just seen way too many false equivocations on western bias vs Chinese bias and may be a bit touchy. Apologies.

http://www.slate.com/blogs/future_tense/2015/01/06/tencent_s_wechat_worldwide_internet_users_are_voluntarily_submitting_to.html
In the last few years, usage of the mobile messaging app WeChat (Weixin), developed by Chinese corporation Tencent, has skyrocketed not only inside China but also around the world. For 500 million mobile users in mainland China, WeChat is one of the only options for mobile messaging available, due to frequent or permanent blockage of apps like WhatsApp, Viber, Line, Twitter, and Facebook. For more than 100 million mobile users in the rest of the world, a highly polished user experience, celebrity marketing, and the promise of “free calls and texts” has proven to be nearly irresistible for far-flung members of the Chinese diaspora. This global user base also includes the Tibetan exile diaspora, who through WeChat have become connected on both sides of the Himalayas in near real time like never before.

Beijing Hinders Free Speech in America: https://www.nytimes.com/2017/11/26/opinion/beijing-free-speech-america.html
unaffiliated  broad-econ  rhetoric  critique  comparison  trends  usa  china  asia  rot  zeitgeist  religion  christianity  theos  morality  values  the-great-west-whale  identity-politics  class  elite  vampire-squid  gender  sex  sinosphere  orient  occident  integrity  multi  gnxp  scitariat  commentary  civilization  counter-revolution  poast  discussion  crooked  capital  finance  cycles  cynicism-idealism  truth  absolute-relative  expansionism  sulla  honor  pessimism  wonkish  diaspora  data  scale  internet  mobile  intel  news  org:lite  world  org:rec  higher-ed  migration  nationalism-globalism  vitality  qra  q-n-a  dominant-minority
july 2017 by nhaliday
What is the difference between taming and domestication? - Quora
Taming is behavioral modification. Domestication is genetic modification.
q-n-a  qra  comparison  jargon  domestication  nature  bio
june 2017 by nhaliday
How important was colonial trade for the rise of Europe? | Economic Growth in History
The latter view became the orthodoxy among economists and economic historians after Patrick O’Brien’s 1982 paper, which in one of many of Patrick’s celebrated phrases, claims that “”the periphery vs peripheral” for Europe. He concludes the paper by writing:

“[G]rowth, stagnation, and decay everywhere in Western Europe can be explained mainly by reference to endogenous forces. … for the economic growth of the core, the periphery was peripheral.”

This is the view that remarkable scholars such as N. Crafts, Deirdre McCloskey, or Joel Mokyr repeat today (though Crafts would argue cotton imports would have mattered in a late stage, and my reading of Mokyr is that he has softened his earlier view from the 1980s a little, specifically in the book The Enlightened Economy.) Even recently, Brad deLong has classifyied O’Brien’s 1982 position as “air tight”.

Among economists and economic historians more on the economics side, I would say that O’Brien’s paper was only one of two strong hits against the “Worlds-System” and related schools of thoughts of the 1970s, the other hit being Solow’s earlier conclusion that TFP growth (usually interpreted as technology, though there’s more to it than that) has accounted for economic growth a great deal more than capital accumulation, which is what Hobsbawm and Wallerstein, in their neo-Marxist framework, emphasize.

https://twitter.com/tcjfs/status/890034395456974848
A friend tonight, on the third world and the first world, and our relationships to the past: "They don't forget, and we don't remember."
https://twitter.com/edwest/status/872337163458932736
imo the European Intifada is being fueled by anti-Europeanism & widely taught ideas like this one discussed - Europe stole its riches

https://www.thinkpragati.com/opinion/1863/dont-blame-empire/
The British Empire was cruel, rapacious and racist. But contrary to what Shashi Tharoor writes in An Era Of Darkness, the fault for India’s miseries lies upon itself.

Indeed, the anti-Tharoor argument is arguably closer to the truth, because the British tended to use the landlord system in places where landlords were already in place, and at times when the British were relatively weak and couldn’t afford to upset tradition. Only after they became confident in their power did the British start to bypass the landlord class and tax the cultivators directly. King’s College London historian Jon Wilson (2016) writes in India Conquered, “Wherever it was implemented, raiyatwar began as a form of military rule.” Thus the system that Tharoor implicitly promotes, and which is associated with higher agricultural productivity today, arose from the very same colonialism that he blames for so many of India’s current woes. History does not always tell the parables that we wish to hear.

...

India’s share of the world economy was large in the eighteenth century for one simple reason: when the entire world was poor, India had a large share of the world’s population. India’s share fell because with the coming of the Industrial Revolution, Europe and North America saw increases of income per capita to levels never before seen in all of human history. This unprecedented growth cannot be explained by Britain’s depredations against India. Britain was not importing steam engines from India.

The big story of the Great Divergence is not that India got poorer, but that other countries got much richer. Even at the peak of Mughal wealth in 1600, the best estimates of economic historians suggest that GDP per capita was 61% higher in Great Britain. By 1750–before the battle of Plassey and the British takeover–GDP per capita in Great Britain was more than twice what it was in India (Broadberry, Custodis, and Gupta 2015). The Great Divergence has long roots.

Tharoor seems blinded by the glittering jewels of the Maharajas and the Mughals. He writes with evident satisfaction that when in 1615 the first British ambassador presented himself to the court of Emperor Jehangir in Agra, “the Englishman was a supplicant at the feet of the world’s mightiest and most opulent monarch.” True; but the Emperor’s opulence was produced on the backs of millions of poor subjects. Writing at the same time and place, the Dutch merchant Francisco Pelsaert (1626) contrasted the “great superfluity and absolute power” of the rich with “the utter subjection and poverty of the common people–poverty so great and miserable that the life of the people can be depicted…only as the home of stark want and the dwelling-place of bitter woe.” Indian rulers were rich because the empire was large and inequality was extreme.

In pre-colonial India the rulers, both Mughal and Maratha, extracted _anywhere from one-third to one half of all gross agricultural output_ and most of what was extracted was spent on opulence and the armed forces, not on improving agricultural productivity (Raychaudhuri 1982).

...

The British were awful rulers but the history of India is a long story of awful rulers (just as it is for most countries). Indeed, by Maddison’s (2007) calculations _the British extracted less from the Indian economy than did the Mughal Dynasty_. The Mughals built their palaces in India while the British built most of their palaces in Britain, but that was little comfort to the Indian peasant who paid for both. The Kohinoor diamond that graces the cover of Inglorious Empire is a telling symbol. Yes, it was stolen by the British (who stole it from the Sikhs who stole it from the Afghanis who stole it from the Mughals who stole it from one of the kings of South India). But how many Indians would have been better off if this bauble had stayed in India? Perhaps one reason why more Indians didn’t take up arms against the British was that for most of them, British rule was a case of meet the new boss, same as the old boss.

more for effect on colonies: https://pinboard.in/u:nhaliday/b:4b0128372fe9

INDIA AND THE GREAT DIVERGENCE: AN ANGLO-INDIAN COMPARISON OF GDP PER CAPITA, 1600-1871: http://eh.net/eha/wp-content/uploads/2013/11/Guptaetal.pdf
This paper provides estimates of Indian GDP constructed from the output side for the pre-1871 period, and combines them with population estimates to track changes in living standards. Indian per capita GDP declined steadily during the seventeenth and eighteenth centuries before stabilising during the nineteenth century. As British living standards increased from the mid-seventeenth century, India fell increasingly behind. Whereas in 1600, Indian per capita GDP was over 60 per cent of the British level, by 1871 it had fallen to less than 15 per cent. As well as placing the origins of the Great Divergence firmly in the early modern period, the estimates suggest a relatively prosperous India at the height of the Mughal Empire, with living standards well above bare bones subsistence.

https://twitter.com/pseudoerasmus/status/832288984009207810
but some of the Asian wage data (especialy India) have laughably small samples (see Broadberry & Gupta)

How profitable was colonialism for various European powers?: https://www.reddit.com/r/AskHistorians/comments/p1q1q/how_profitable_was_colonialism_for_various/

How did Britain benefit from colonising India? What did colonial powers gain except for a sense of power?: https://www.quora.com/How-did-Britain-benefit-from-colonising-India-What-did-colonial-powers-gain-except-for-a-sense-of-power
The EIC period was mostly profitable, though it had recurring problems with its finances. The initial voyages from Surat in 1600s were hugely successful and brought profits as high as 200%. However, the competition from the Dutch East India Company started to drive down prices, at least for spices. Investing in EIC wasn’t always a sure shot way to gains - British investors who contributed to the second East India joint stock of 1.6 million pounds between 1617 and 1632 ended up losing money.

...

An alternate view is that the revenues of EIC were very small compared to the GDP of Britain, and hardly made an impact to the overall economy. For instance, the EIC Revenue in 1800 was 7.8m pounds while the British GDP in the same period was 343m pounds, and hence EIC revenue was only 2% of the overall GDP. (I got these figures from an individual blog and haven’t verified them).

...

The British Crown period - The territory of British India Provinces had expanded greatly and therefore the tax revenues had grown in proportion. The efficient taxation system paid its own administrative expenses as well as the cost of the large British Indian Army. British salaries were lucrative - the Viceroy received £25,000 a year, and Governors £10,000 for instance besides the lavish amenities in the form of subsidized housing, utilities, rest houses, etc.

...

Indian eminent intellectual, Dadabhai Naoroji wrote how the British systematically ensured the draining of Indian economy of its wealth and his theory is famously known as ‘Drain of Wealth’ theory. In his book 'Poverty' he estimated a 200–300 million pounds loss of revenue to Britain that is not returned.

At the same time, a fair bit of money did go back into India itself to support further colonial infrastructure. Note the explosion of infrastructure (Railway lines, 100+ Cantonment towns, 60+ Hill stations, Courthouses, Universities, Colleges, Irrigation Canals, Imperial capital of New Delhi) from 1857 onward till 1930s. Of course, these infrastructure projects were not due to any altruistic motive of the British. They were intended to make their India empire more secure, comfortable, efficient, and to display their grandeur. Huge sums of money were spent in the 3 Delhi Durbars conducted in this period.

So how profitable was the British Crown period? Probably not much. Instead bureaucracy, prestige, grandeur, comfort reigned supreme for the 70,000 odd British people in India.

...

There was a realization in Britain that colonies were not particularly economically beneficial to the home economy. … [more]
econotariat  broad-econ  article  history  early-modern  age-of-discovery  europe  the-great-west-whale  divergence  conquest-empire  economics  growth-econ  roots  trade  endo-exo  patho-altruism  expansionism  multi  twitter  social  discussion  gnon  unaffiliated  right-wing  🎩  attaq  albion  journos-pundits  mokyr-allen-mccloskey  cjones-like  big-picture  chart  news  org:mag  org:foreign  marginal-rev  wealth-of-nations  britain  india  asia  cost-benefit  leviathan  antidemos  religion  islam  class  pop-structure  nationalism-globalism  authoritarianism  property-rights  agriculture  econ-metrics  data  scale  government  industrial-revolution  pdf  regularizer  pseudoE  measurement  volo-avolo  time-series  anthropology  macro  sapiens  books  review  summary  counterfactual  stylized-facts  critique  heavy-industry  pre-ww2  study  technology  energy-resources  labor  capitalism  debate  org:data  org:lite  commentary  usa  piketty  variance-components  automation  west-hunter  scitariat  visualization  northeast  the-south  aphorism  h2o  fluid
june 2017 by nhaliday
How is Chinese history organized? - Quora
Typically, in my mind, this is how I structure history:
1. Dinosaurs
2. Cavemen
3. Pre-Christian
4. Roman (500 BC to 500 AD)
5. Dark Ages
6. Middle Ages
7. Renaissance (Machiavelli and the Tudors go here.)
8. Reformation
9. Industrial Age (WWI & WWII go here.)
10. Information Age

First two are the same.
3. Dawn of civilization.
4. Loose united, feudal lords fighting each other.
5. United by first emperor of Qin, centralised government.
6. Warlords fighting for uniting.
7. United.
8. Repeat 6&7 for several times.
9. Last empire Qing.
10. Moderation, struggling from foreign invading force, revolution.
11. Revolution of revolution, communism take over.
12. Reform, non-communism.
13. Economic powerhouse, world factory.
q-n-a  qra  skeleton  thinking  big-picture  summary  history  antiquity  iron-age  medieval  early-modern  mostly-modern  china  asia  sinosphere  comparison  civilization  canon  time  sequential  chart
june 2017 by nhaliday
mandarin - Is it easier to learn Chinese after learning Japanese or vice versa? - Chinese Language Stack Exchange
Apart from the Kanji/Hanzi, that they (partly) have in common, concerning the written part, there is nothing that can really help you with the other language:
- Chinese is pretty much SVO, Japanese is SOV;
- Chinese has tones, Japanese has no tones. When speaking, sentences do have a certain "tone", but not phonemic, i.e. it doesn't totally change the meaning;
- Chinese has one writing system (Hanzi), Japanese has 3 (Hiragana, Katakana, Kanji);

https://www.quora.com/Is-it-easier-to-learn-Japanese-after-youve-obtained-proficiency-in-Chinese-as-a-Westerner
https://www.quora.com/How-difficult-is-is-to-learn-Japanese-for-a-native-Chinese-speaker
q-n-a  stackex  world  foreign-lang  language  china  asia  japan  sinosphere  multi  qra  direct-indirect  syntax  learning  generalization  universalism-particularism
june 2017 by nhaliday
Erik Archer's answer to Are Japanese people impressed by a foreigner who can fluently speak Japanese? - Quora
“Yes” at first, “no” in the end and it kind of breaks down into this ideology “Japanese people love it when foreigners visit, but hate it when they stay.
q-n-a  qra  japan  asia  culture  migration  language  foreign-lang
june 2017 by nhaliday
Could you explain the character of Fat Tony in Antifragile by Taleb? - Quora
Dr. John can make gigantic errors that affect other people by ignoring reality in favor of assumptions. Fat Tony makes smaller errors that affect only himself, but more seriously (they kill him).
q-n-a  qra  aphorism  jargon  analogy  narrative  blowhards  outcome-risk  noise-structure
may 2017 by nhaliday
In the first place | West Hunter
We hear a lot about innovative educational approaches, and since these silly people have been at this for a long time now, we hear just as often about the innovative approaches that some idiot started up a few years ago and are now crashing in flames.  We’re in steady-state.

I’m wondering if it isn’t time to try something archaic.  In particular, mnemonic techniques, such as the method of loci.  As far as I know, nobody has actually tried integrating the more sophisticated mnemonic techniques into a curriculum.  Sure, we all know useful acronyms, like the one for resistor color codes, but I’ve not heard of anyone teaching kids how to build a memory palace.

https://westhunt.wordpress.com/2013/12/28/in-the-first-place/#comment-20106
I have never used formal mnemonic techniques, but life has recently tested me on how well I remember material from my college days. Turns out that I can still do the sorts of math and physics problems that I could then, in subjects like classical mechanics, real analysis, combinatorics, complex variables, quantum mechanics, statistical mechanics, etc. I usually have to crack the book though. Some of that material I have used from time to time, or even fairly often (especially linear algebra), most not. I’m sure I’m slower than I was then, at least on the stuff I haven’t used.

https://westhunt.wordpress.com/2013/12/28/in-the-first-place/#comment-20109
Long-term memory capacity must be finite, but I know of no evidence that anyone has ever run out of it. As for the idea that you don’t really need a lot of facts in your head to come up with new ideas: pretty much the opposite of the truth, in a lot of fields.

https://en.wikipedia.org/wiki/Method_of_loci

Mental Imagery > Ancient Imagery Mnemonics: https://plato.stanford.edu/entries/mental-imagery/ancient-imagery-mnemonics.html
In the Middle Ages and the Renaissance, very elaborate versions of the method evolved, using specially learned imaginary spaces (Memory Theaters or Palaces), and complex systems of predetermined symbolic images, often imbued with occult or spiritual significances. However, modern experimental research has shown that even a simple and easily learned form of the method of loci can be highly effective (Ross & Lawrence, 1968; Maguire et al., 2003), as are several other imagery based mnemonic techniques (see section 4.2 of the main entry).

The advantages of organizing knowledge in terms of country and place: http://marginalrevolution.com/marginalrevolution/2018/02/advantages-organizing-knowledge-terms-country-place.html

https://www.quora.com/What-are-the-best-books-on-Memory-Palace

fascinating aside:
US vs Nazi army, Vietnam, the draft: https://westhunt.wordpress.com/2013/12/28/in-the-first-place/#comment-20136
You think I know more about this than a retired major general and former head of the War College? I do, of course, but that fact itself should worry you.

He’s not all wrong, but a lot of what he says is wrong. For example, the Germany Army was a conscript army, so conscription itself can’t explain why the Krauts were about 25% more effective than the average American unit. Nor is it true that the draft in WWII was corrupt.

The US had a different mix of armed forces – more air forces and a much larger Navy than Germany. Those services have higher technical requirements and sucked up a lot of the smarter guys. That was just a product of the strategic situation.

The Germans had better officers, partly because of better training and doctrine, partly the fruit of a different attitude towards the army. The US, much of the time, thought of the Army as a career for losers, but Germans did not.

The Germans had an enormous amount of relevant combat experience, much more than anyone in the US. Spend a year or two on the Eastern Front and you learn.

And the Germans had better infantry weapons.

The US tooth-to-tail ratio was , I think, worse than that of the Germans: some of that was a natural consequence of being an expeditionary force, but some was just a mistake. You want supply sergeants to be literate, but it is probably true that we put too many of the smarter guys into non-combat positions. That changed some when we ran into manpower shortages in late 1944 and combed out the support positions.

This guy is back-projecting Vietnam problems into WWII – he’s mostly wrong.

more (more of a focus on US Marines than Army): https://www.quora.com/Were-US-Marines-tougher-than-elite-German-troops-in-WW2/answer/Joseph-Scott-13
west-hunter  scitariat  speculation  ideas  proposal  education  learning  retention  neurons  the-classics  nitty-gritty  visuo  spatial  psych-architecture  multi  poast  history  mostly-modern  world-war  war  military  strategy  usa  europe  germanic  cold-war  visual-understanding  cartoons  narrative  wordlessness  comparison  asia  developing-world  knowledge  metabuch  econotariat  marginal-rev  discussion  world  thinking  government  local-global  humility  wire-guided  policy  iron-age  mediterranean  wiki  reference  checklists  exocortex  early-modern  org:edu  philosophy  enlightenment-renaissance-restoration-reformation  qra  q-n-a  books  recommendations  list  links  ability-competence  leadership  elite  higher-ed  math  physics  linear-algebra  cost-benefit  prioritizing  defense  martial  war-nerd  worrydream
may 2017 by nhaliday
per page:    204080120160

Copy this bookmark:

to read