recentpopularlog in

jerryking : economic_data   10

Algos know more about us than we do about ourselves
NOVEMBER 24, 2017 | Financial Time | John Dizard.

When intelligence collectors and analysts take an interest in you, they usually start not by monitoring the content of your calls or messages, but by looking at the patterns of your communications. Who are you calling, how often and in what sequence? What topics do you comment on in social media?

This is called traffic analysis, and it can give a pretty good notion of what you and the people you know are thinking and what you are preparing to do. Traffic analysis started as a military intelligence methodology, and became systematic around the first world war. Without even knowing the content of encrypted messages, traffic analysts could map out an enemy “order of battle” or disposition of forces, and make inferences about commanders’ intentions.

Traffic analysis techniques can also cut through the petabytes of redundant babble and chatter in the financial and political worlds. Even with state secrecy and the forests of non-disclosure agreements around “proprietary” investment or trading algorithms, crowds can be remarkably revealing in their open-source posts on social media.

Predata, a three-year-old New York and Washington-based predictive data analytics provider, has a Princeton-intensive crew of engineers and international affairs graduates working on early “signals” of market and political events. Predata trawls the open metadata for users of Twitter, Wikipedia, YouTube, Reddit and other social media, and analyses it to find indicators of future price moves or official actions.

I have been following their signals for a while and find them to be useful indicators. Predata started by creating political risk indicators, such as Iran-Saudi antagonism, Italian or Chilean labour unrest, or the relative enthusiasm for French political parties. Since the beginning of this year, they have been developing signals for financial and commodities markets.

The 1-9-90 rule
1 per cent of internet users initiate discussions or content, 9 per cent transmit content or participate occasionally and 90 per cent are consumers or ‘lurkers’

Using the example of the company’s BoJ signal. For this, Predata collects the metadata from 300 sources, such as Twitter users, contested Wikipedia edits or YouTube items created by Japanese monetary policy geeks. Of those, at any time perhaps 100 are important, and 8 to 10 turn out to be predictive....This is where you need some domain knowledge [domain expertise = industry expertise]. It turns out that Twitter is pretty important for monetary policy, along with the Japanese-language Wiki page for the Bank of Japan, or, say, a YouTube video of [BoJ governor] Haruhiko Kuroda’s cross-examination before a Diet parliamentary committee.

“Then you build a network of candidate discussions [JK: training beds] and look for the pattern those took before historical moves. The machine-learning algorithm goes back and picks the leads and lags between traffic and monetary policy events.” [Jk: Large data sets with known correct answers serve as a training bed and then new data serves as a test bed]

Typically, Predata’s algos seem to be able to signal changes in policy or big price moves [jk: inflection points] somewhere between 2 days and 2 weeks in advance. Unlike some academic Twitter scholars, Predata does not do systematic sentiment analysis of tweets or Wikipedia edits. “We only look for how many people there are in the conversation and comments, and how many people disagreed with each other. We call the latter the coefficient of contestation,” Mr Shinn says.

The lead time for Twitter, Wiki or other social media signals varies from one market to another. Foreign exchange markets typically move within days, bond yields within a few days to a week, and commodities prices within a week to two weeks. “If nothing happens within 30 days,” says Mr Lee, “then we say we are wrong.”
algorithms  alternative_data  Bank_of_Japan  commodities  economics  economic_data  financial_markets  industry_expertise  inflection_points  intelligence_analysts  lead_time  machine_learning  massive_data_sets  metadata  non-traditional  Predata  predictive_analytics  political_risk  signals  social_media  spycraft  traffic_analysis  training_beds  Twitter  unconventional 
november 2017 by jerryking
Steve Ballmer Serves Up a Fascinating Data Trove - The New York Times
Andrew Ross Sorkin
Continue reading the main storyShare This Page

Steve_Ballmer  government  Andrew_Sorkin  databases  data  measurements  economics  indicators  real-time  forecasting  economic_data 
april 2017 by jerryking
What Scented Candles Say to an Economist - The New York Times

We need a wider variety of indicators to help us take a more accurate reading of the economy. Some of these might seem frivolous, but paying close attention to worldly detail could make forecasting more reliable.
(1) height of hemlines
(2) the number of cranes visible on the skyline
(3) Spending on luxury items is another example. During a boom, sales of fast cars, expensive paintings, prime real estate and diamond necklaces all soar, as do their prices.

Less obvious are trends in retailing. When the good times roll, people decide that their great idea for a specialty store is viable. Thus booms bring all those boutiques selling just one type of good: socks or scented candles or freshly squeezed juices. But like flowers that display the behavior known as nyctinasty — opening to the sun’s light and warmth — they close as soon as the skies darken and things start to cool.

(4) how easy, or otherwise, it is to get restaurant reservations or tickets for shows.
(5) how many “help wanted” signs appear in the windows of stores and restaurants.

....G.D.P. almost certainly fails to capture newer areas of economic activity, such as today’s digital innovation — so other sources of information are needed to fill the gap....economic policy makers usually scrutinize tens, or even hundreds, of indicators, covering different industries and assets, different parts of the country, different groups of people. They monitor jobs reports, advertising rates, wage settlements, the cost of shipping freight, asset prices, sales of consumer durables and much, much more.
economics  economists  forecasting  non-obvious  GDP  indicators  trends  retailers  boutiques  detail_oriented  economic_data  information_sources  policymakers  policymaking 
november 2015 by jerryking
More Data Can Mean Less Guessing About the Economy -
Published: September 7, 2013

measurement shortfall in the small-business sector, and a series of other information gaps in the economy, may be overcome by what experts say is an emerging data revolution — Big Data, in the current catchphrase. The ever-expanding universe of digital signals of behavior, from browsing and buying on the Web to cellphone location data, is grist for potential breakthroughs in economic measurement. It could produce more accurate forecasting and more informed policy-making — more science and less guesswork.... THE economics profession is gearing up to exploit new sources of digital data. In a recent paper, “The Data Revolution and Economic Analysis,” two Stanford economists, Liran Einav and Jonathan Levin, concluded that “there is little doubt, at least in our minds, that over the next decades ‘big data’ will change the landscape of economic policy and economic research.”

At Intuit, the small-business data portray a sector that was “hurt much more than big business by the recession and its recovery has been far worse,” says Ms. Woodward, the economic consultant. Over the last three and a half years, payroll employment for all companies has increased 6.9 percent, while small-business employment has risen far less, just 1.9 percent. Hiring among the small companies, though still sluggish, has inched ahead in the last three months.
data  Steve_Lohr  massive_data_sets  Intuit  information_sources  small_business  measurements  Freshbooks  economy  Erik_Brynjolfsson  economics  indicators  real-time  forecasting  economic_data  information_gaps  signals  economists  data_driven 
september 2013 by jerryking
The Financial Bonanza of Big Data
Vast troves of information are manipulated and monetized, yet companies have a hard time assigning value to it...The value of information captured today is increasingly in the myriad secondary uses to which it is put—not just the primary purpose for which it was collected.[True, but this secondary or exhaust data has to be placed in the right context in order to maximize value]. In the past, shopkeepers kept a record of all transactions so that they could tally the sums at the end of the day. The sales data were used to understand sales. Only more recently have retailers parsed those records to look for business trends...With big data, information is more potent, and it can be applied to areas unconnected with what it initially represented. Health officials could use Google's history of search queries—for things like cough syrup or sneezes—to track the spread of the seasonal flu in the United States. The Bank of England has used Google searches as a leading indicator for housing prices in the United Kingdom. Other central banks have studied search queries as a gauge for changes in unemployment.

Companies world-wide are starting to understand that no matter what industry they are in, data is among their most precious assets. Harnessed cleverly, the data can unleash new forms of economic value.
massive_data_sets  Amazon  books  Google  branding  Facebook  Wal-Mart  Bank_of_England  data  data_driven  value_creation  JCK  exhaust_data  commercialization  monetization  valuations  windfalls  alternative_data  economic_data  tacit_data  interpretation  contextual  sense-making  tacit_knowledge 
march 2013 by jerryking
Google to map inflation using web data
October 11 2010 | Financial Times | By Robin Harding in
Denver. Google is using its vast database of web shopping data to
construct the ‘Google Price Index’ – a daily measure of inflation that
could one day provide an alternative to official statistics
The work by Google’s chief economist, Hal Varian, highlights how
economic data can be gathered far more rapidly using online sources. The
official Consumer Price Index data are collected by hand from shops,
and only published monthly with a time lag of several weeks.
Hal_Varian  Google  inflation  statistics  mapping  massive_data_sets  economic_data  CPI 
october 2010 by jerryking
Nigel Wright is cut from different cloth
Sept. 24, 2010 | Globe & Mail | Andrew Steele.

So what of Mr. Wright and his ability to manage this agenda? For starters, his background is unusual. World-class financial dealmakers are not the normal cloth from which one cuts a political aide.

Since Jack Pickersgill invented the role for Mackenzie King, Canadian chiefs of staff have been smart political operators like Jean Pelletier or Tim Murphy or Hugh Segal or Marc Lalonde or Tom Kent. Often lawyers. Sometimes academics. Sometimes seconded civil servants.......The appointment of Nigel Wright as chief of staff is worth some serious
discussion, because it is a very shrewd move by the PM. Basically, the
PM’s chief of staff has 3 major challenges. 1. The PM must focus his
energies – in public and in private – almost exclusively to the economy.
Incumbents across the continent are in peril because of the lingering
impacts of the recession: unemployment primarily, but also the threats
of inflation, interest rates, credit crunches, real estate devaluation
and sovereign debt. 2. The PM only gets to focus on the economy if all
the other issues are managed down. 3. The only message out of Ottawa
most days will need to be about the economy and what the PM is doing
about it. Mr. Harper is clearly aware that the economy is his biggest
threat. With Nigel Wright, he is making a move that continues to address
that threat.
Stephen_Harper  Nigel_Wright  Onex  private_equity  chief_of_staff  economic_downturn  economic_data  debt 
september 2010 by jerryking
New Economists Scour Urban Data for Trends -
APRIL 8, 2010 | Wall Street Journal | by CARI TUNA. New Ways
to Read Economy. Experts Scour Oddball Data to Help See Trends Before
Official Information Is Available. A growing number of economists and
urban planners [are] scouring for economic clues in unconventional urban
data—oddball measures of how people are moving, spending and working.
"Mr. Egan said he would like to build software to monitor Craigslist
prices for furniture, concert tickets, haircuts and other goods and
services to measure changes in local prices. The online classified-ads
site, he said, would give a quicker and more detailed read than the
bimonthly data from the Labor Department."

Broadway ticket sales are a favorite indicator for the chief economist of the New York City Economic Development Corp., Francesco Brindisi. He says they are a good gauge of city tourism.

In Jacksonville, Fla., community planner Ben Warner keeps tabs on calls to the city's 2-1-1 hotline for social services. Since late 2008, he has seen spikes in calls for help with food, housing, utilities payments and suicide prevention. It is "direct, real-time monitoring of the economic and social situation," he said.
data  urban  unconventional_thinking  economic_analyses  craigslist  Hal_Varian  hotlines  massive_data_sets  Freshbooks  economists  trends  pattern_recognition  measurements  real-time  forecasting  indicators  unorthodox  economic_data  metrics  Cari_Tuna  data_driven  unconventional  economics  non-traditional 
april 2010 by jerryking
Annals of Innovation: How David Beats Goliath: Reporting & Essays: The New Yorker
May 11, 2009 |The New Yorker | by Malcolm Gladwell. How
underdogs create opportunities by first understanding their strengths,
weaknesses, and the rules of the game, and then changing the rules....To Gladwell, the story illustrated how traditions become blind spots. “Playing insurgent basketball did not guarantee victory. It was simply the best chance an underdog had of beating Goliath,” he wrote. “And yet somehow that lesson has escaped the basketball establishment.” The anecdote became the opening passage of the book David and Goliath, another fixture on bestseller lists....A few years ago, Ranadivé wrote a paper arguing that even the Federal Reserve ought to make its decisions in real time—not once every month or two. “Everything in the world is now real time,” he said. “So when a certain type of shoe isn’t selling at your corner shop, it’s not six months before the guy in China finds out. It’s almost instantaneous, thanks to my software. The world runs in real time, but government runs in batch. Every few months, it adjusts. Its mission is to keep the temperature comfortable in the economy, and, if you were to do things the government’s way in your house, then every few months you’d turn the heater either on or off, overheating or underheating your house.” Ranadivé argued that we ought to put the economic data that the Fed uses into a big stream, and write a computer program that sifts through those data, the moment they are collected, and make immediate, incremental adjustments to interest rates and the money supply. “It can all be automated,” he said. “Look, we’ve had only one soft landing since the Second World War. Basically, we’ve got it wrong every single time.”
anecdotal  basketball  batch_processing  blind_spots  books  coaching  creating_opportunities  decision_making  economic_data  innovation  interest_rates  Malcolm_Gladwell  massive_data_sets  money_supply  overlooked_opportunities  rainmaking  real-time  rules_of_the_game  strategy  strengths  Tibco  underdogs  U.S._Federal_Reserve  Vivek_Ranadivé  weaknesses 
may 2009 by jerryking

Copy this bookmark:

to read