recentpopularlog in

jerryking : metadata   28

The Mystery of the Miserable Employees: How to Win in the Winner-Take-All Economy -
June 15, 2019 | The New York Times | By Neil Irwin.
Neil Irwin is a senior economics correspondent for The Upshot. He is the author of “How to Win in a Winner-Take-All-World,” a guide to navigating a career in the modern economy.......
What Mr. Ostrum and the analytics team did wasn’t a one-time dive into the numbers. It was part of a continuing process, a way of thinking that enabled them to change and adapt along with the business environment. The key is to listen to what data has to say — and develop the openness and interpretive skills to understand what it is telling us.......Neil Irwin was at Microsoft’s headquarters researching a book that aims to answer one simple question: How can a person design a thriving career today? The old advice (show up early, work hard) is no longer enough....In nearly every sector of the economy, people who seek well-paying, professional-track success face the same set of challenges: the rise of a handful of dominant “superstar” firms; a digital reinvention of business models; and a rapidly changing understanding about loyalty in the employer-employee relationship. It’s true in manufacturing and retail, in banking and law, in health care and education — and certainly in tech......superstar companies — and the smaller firms seeking to upend them — are where pragmatic capitalists can best develop their abilities and be well compensated for them over a long and durable career.....the obvious disadvantages of bureaucracy have been outweighed by some not-so-obvious advantages of scale......the ability to collect and analyze vast amounts of data about how people work, and what makes a manager effective (jk: organizing data) .... is essential for even those who aren’t managers of huge organizations, but are just trying to make themselves more valuable players on their own corporate team.......inside Microsoft’s human resources division, a former actuary named Dawn Klinghoffer ....was trying to figure out if the company could use data about its employees — which ones thrived, which ones quit, and the differences between those groups — to operate better......Klinghoffer was frustrated that ....insights came mostly from looking through survey results. She was convinced she could take the analytical approach further. After all, Microsoft was one of the biggest makers of email and calendar software — programs that produce a “digital exhaust” of metadata about how employees use their time. In September 2015, she advised Microsoft on the acquisition of a Seattle start-up, VoloMetrix, that could help it identify and act on the patterns in that vapor......One of VoloMetrix's foundational data sets, for example, was private emails sent by top Enron executives before the company’s 2001 collapse — a rich look at how an organization’s elite behave when they don’t think anyone is watching.
analytics  books  data  datasets  data_driven  exhaust_data  Fitbit  gut_feelings  human_resources  interpretative  Managing_Your_Career  massive_data_sets  meetings  metadata  Microsoft  Moneyball  organizational_analytics  organizing_data  people_analytics  quantitative  quantified_self  superstars  unhappiness  VoloMetrix  winner-take-all  work_life_balance 
june 2019 by jerryking
Algos know more about us than we do about ourselves
NOVEMBER 24, 2017 | Financial Time | John Dizard.

When intelligence collectors and analysts take an interest in you, they usually start not by monitoring the content of your calls or messages, but by looking at the patterns of your communications. Who are you calling, how often and in what sequence? What topics do you comment on in social media?

This is called traffic analysis, and it can give a pretty good notion of what you and the people you know are thinking and what you are preparing to do. Traffic analysis started as a military intelligence methodology, and became systematic around the first world war. Without even knowing the content of encrypted messages, traffic analysts could map out an enemy “order of battle” or disposition of forces, and make inferences about commanders’ intentions.

Traffic analysis techniques can also cut through the petabytes of redundant babble and chatter in the financial and political worlds. Even with state secrecy and the forests of non-disclosure agreements around “proprietary” investment or trading algorithms, crowds can be remarkably revealing in their open-source posts on social media.

Predata, a three-year-old New York and Washington-based predictive data analytics provider, has a Princeton-intensive crew of engineers and international affairs graduates working on early “signals” of market and political events. Predata trawls the open metadata for users of Twitter, Wikipedia, YouTube, Reddit and other social media, and analyses it to find indicators of future price moves or official actions.

I have been following their signals for a while and find them to be useful indicators. Predata started by creating political risk indicators, such as Iran-Saudi antagonism, Italian or Chilean labour unrest, or the relative enthusiasm for French political parties. Since the beginning of this year, they have been developing signals for financial and commodities markets.

The 1-9-90 rule
1 per cent of internet users initiate discussions or content, 9 per cent transmit content or participate occasionally and 90 per cent are consumers or ‘lurkers’

Using the example of the company’s BoJ signal. For this, Predata collects the metadata from 300 sources, such as Twitter users, contested Wikipedia edits or YouTube items created by Japanese monetary policy geeks. Of those, at any time perhaps 100 are important, and 8 to 10 turn out to be predictive....This is where you need some domain knowledge [domain expertise = industry expertise]. It turns out that Twitter is pretty important for monetary policy, along with the Japanese-language Wiki page for the Bank of Japan, or, say, a YouTube video of [BoJ governor] Haruhiko Kuroda’s cross-examination before a Diet parliamentary committee.

“Then you build a network of candidate discussions [JK: training beds] and look for the pattern those took before historical moves. The machine-learning algorithm goes back and picks the leads and lags between traffic and monetary policy events.” [Jk: Large data sets with known correct answers serve as a training bed and then new data serves as a test bed]

Typically, Predata’s algos seem to be able to signal changes in policy or big price moves [jk: inflection points] somewhere between 2 days and 2 weeks in advance. Unlike some academic Twitter scholars, Predata does not do systematic sentiment analysis of tweets or Wikipedia edits. “We only look for how many people there are in the conversation and comments, and how many people disagreed with each other. We call the latter the coefficient of contestation,” Mr Shinn says.

The lead time for Twitter, Wiki or other social media signals varies from one market to another. Foreign exchange markets typically move within days, bond yields within a few days to a week, and commodities prices within a week to two weeks. “If nothing happens within 30 days,” says Mr Lee, “then we say we are wrong.”
algorithms  alternative_data  Bank_of_Japan  commodities  economics  economic_data  financial_markets  industry_expertise  inflection_points  intelligence_analysts  lead_time  machine_learning  massive_data_sets  metadata  non-traditional  Predata  predictive_analytics  political_risk  signals  social_media  spycraft  traffic_analysis  training_beds  Twitter  unconventional 
november 2017 by jerryking
Small Data: Why Tinder-like apps are the way of the future — Medium
+++++++++++++++++++++++
The card-based UI updates the classic way in which we’ve always interacted with physical cards. When you think about it, cards are nothing more than bite-size presentations of concrete information. They’re the natural evolution of the newsfeed, which is useful for reading stories but not for making decisions.
++++++++++++++++++++++++
Cards are kind of natural choice for mobile screens because of their size and shape. But lay your cards on the table or put them on a board and they will also help you in revealing connections, understanding the topic and making decisions.
++++++++++++++++++++++++

every single interaction with card-swiping apps can affect the outcome.

We can call it small data. Imagine if every time you made a yes or no decision on Tinder, the app learned what kind of profiles you tended to like, and it showed you profiles based on this information in the future.

“With swipes on Tinder, the act of navigating through content is merged with inputting an action on that content,” says Rad. That means that every time a user browses profiles, it generates personal behavioral data.
bite-sized  Tinder  small_data  ux  design  decision_making  information_overload  behavioural_data  metadata  gestures  Snapchat  personal_data 
march 2014 by jerryking
Using 'remarkable' source of data, startup builds rich customer profiles - The Globe and Mail
Ivor Tossell

Special to The Globe and Mail

Published Monday, Jan. 06 2014

RetailGenius, a product from a Toronto startup called Viasense, promises to algorithmically generate customer profiles based on a remarkable source of data: Anonymous location data that’s collected by big mobile carriers, from the passive pings that every single cellphone sends out as it goes through the day.

The data that RetailGenius uses is anonymized – it doesn’t have any way of knowing whose cellphone belongs to who; it simply has a gigantic plot of where thousands of cellphones were at any given time.

“We create a unique identifier between those signals, and we can see those signals move throughout the city,” says Mossab Basir, RetailGenius’ founder. “We can see those changes in your location but we never really know who it is.”

What the product does next is intriguing: Based on some 50 million pieces of location data a day, RetailGenius crunches the numbers to make inferences from where each cellphone spends its time, and generates customer profiles by the thousands.

For instance, if a given cellphone spends the hours between 7 p.m. and 6 a.m. in a single area, it’s a good bet that its owner lives there. If that cellphone spends its working hours downtown five days a week, its owner is probably a daily commuter. And if it visits a given retail store once a week, a picture of its owner’s habits living and shopping habits starts to emerge.

By lumping these inferred profiles together, RetailGenius can give retailers a picture of who walks through their doors. For instance: What are the top 50 postal codes that are represented in their customers? What kind of volumes of customers are arriving at the store? How long do they stay?
data  start_ups  customer_insights  customer_profiling  RetailGenius  location_based_services  massive_data_sets  data_marketplaces  algorithms  Viasense  metadata  postal_codes  inferences  information_sources  anonymized  shopping_habits 
january 2014 by jerryking
Bell planning to use customers' data to target ads - The Globe and Mail
Oct. 22 2013 | The Globe and Mail | SUSAN KRASHINSKY.

Bell Canada is planning to use information about its customers’ accounts and Internet use to target ads to them.

The information Bell will be using includes Internet activity from both mobile devices and computers, including Web pages customers have visited and search terms they have entered; customers’ location; use of apps and other device features; television viewing habits; and “calling patterns.” Account information shared will include product use including type of device, payment patterns, language preferences, postal codes, and demographic information.
Susan_Krashinsky  Bell_Canada  data  data_driven  data_mining  demographic_information  massive_data_sets  target_marketing  behavioural_targeting  online_behaviour  metadata 
november 2013 by jerryking
Busy and Busier
Oct 24 2012 | The Atlantic | James Fallows.

a lot of people are feeling overwhelmed is because people are not in true survival or crisis mode as often as they have been in much of our history. The interesting thing about crisis is that it actually produces a type of serenity. Why? Because in a crisis, people have to integrate all kinds of information that’s potentially relevant, they have to make decisions quickly, they have to then trust their intuitive judgment calls in the moment. They have to act. They’re constantly course-correcting based on data that’s coming up, and they’re very focused on some outcome, usually live—you know, survive. Don’t burn up. Don’t die.

But as soon as you’re not in a crisis, all the rest of the world floods into your psyche. Now you’re worried about taxes and tires and “I’m getting a cold” and “My printer just crapped out.” Now that flood is coming across in electronic form, and it is 24/7.....The thing about nature is, it’s information rich, but the meaningful things in nature are relatively few—berries, bears and snakes, thunderstorms, maybe poison oak. There are only a few things in nature that force me to change behavior or make a decision. The problem with e-mail is that it’s not just information; it’s the need for potential action. It’s the berries and snakes and bears, but they’re embedded, and you don’t know what’s in each one....Things on your mind need to be externalized—captured in some system that you trust. You capture things that are potentially meaningful; you clarify what those things mean to you; and you need maps of all that, so you can see it from a larger perspective. With better technology, I’d like a set of maps—maps of my maps. Then I could say, “Okay, which map do I want to work on right now? Do I want to work on my family map, because I’ve got family members coming over for dinner?” Then you can drill down into “Oh, my niece is coming. She likes this food, her favorite color is pink, her dog is named …” Then you can back off and say, “That’s enough of that map. What’s the next map I want to see?” Or: “I’d just like to read some poetry right now.”  [JCK: this is really an example of thinking in layers]
busy_work  course_correction  crisis  David_Allen  GTD  human_psyche  information_overload  James_Fallows  living_in_the_moment  mapping  mental_maps  metacognition  metadata  metaphysical  monotasking  productivity  nature  noise  overwhelmed  self-organization  sense-making  signals  stress_response  thinking  thinking_deliberatively 
november 2013 by jerryking
The slides that came in from Brazil
Oct. 07 2013 | The Globe and Mail |editorials.

Brazil is entitled to an explanation from the Canadian government about what appear to be plans for economic espionage on the Brazilian Ministry of Mines and Energy (and consequently on Brazilian companies) by the Communications Security Establishment Canada. And Canadian citizens are entitled to a clear, principled statement of the views of the CSEC and the Canadian government as a whole on what kinds of economic intelligence they believe themselves to be justified in collecting....CSEC’s signals-intelligence activities should not, as a general rule, be put in the service of private companies, either Canadian or foreign. Canadian competitiveness is of course a desirable goal, but one essential element of fair competition, internationally as well as within a home country, is that it should not be deceptive or fraudulent.

Reports over the years have suggested that CSEC has provided the government with economic intelligence in trade negotiations. If so, the practice is dubious. Trade is not war, and trade negotiations should be carried on in good faith – with elements of strategy on both sides.
Brazil  mining  Canadian  security_&_intelligence  editorials  espionage  cyber_security  CSE  sigint  metadata  GoC 
october 2013 by jerryking
Data Is the World
Aug 1, 2005 | Inc.com | Michael S. Hopkins.

Use your data. "Companies aren't taking advantage of the data they generate, Levitt says. "Often, data generated for one purpose is useful for another. Freakonomics describes the case of an entrepreneur selling bagels in corporate offices who kept meticulous records to track profits and loss—data that eventually yielded insights about white-collar crime and the effects of office size on honesty.
Ask different questions. The abortion-crime link revealed itself when Levitt thought to stop asking what made crime fall and try asking why it had risen so much in the first place. That led him to justice system practices in the 1960s, which led him to a statistical understanding of which individuals were likeliest to commit crimes, and ultimately to the question of why a large segment of that population seemed to have vanished.
Don't mistake correlation for causality. Innovative policing and a drop in crime happened simultaneously, but data proved the one didn't cause the other. (Be mindful of the feudal king who, having learned disease was greatest in regions with the most doctors, figured that reducing doctors would reduce disease.)
Question conventional wisdom. An idea that is both easy to understand and a source of comfort (such as the credit quickly given to innovative policing for cutting crime) should be especially suspect.
Respect the complexity of incentives. "You can't imagine, says Levitt, "all the ways humans will connive to beat a system.
Employ data against cheating. Just as companies don't sufficiently capitalize on the data they have access to, they aren't exploiting what Levitt calls "opportunities to think about fraud or theft in their businesses.
'60s  bank_shots  causality  cheating  conventional_wisdom  correlations  data  data_driven  exhaust_data  Freakonomics  gaming_the_system  incentives  insights  justice_system  massive_data_sets  metadata  oversimplification  questions  skepticism  social_data  Steven_Levitt  theft  think_differently  white-collar_crime 
january 2013 by jerryking
A conversation that translates
June 7, 2012 | The Financial Times pg. 14 | Philip Delves Broughton.
(Pass on to Abdoulaye DIOP)
For global companies, creating an approach to risk that resonates across cultures can be a challenge, writes Philip Delves Broughton

Risk is a risky word. Already prone to misinterpretation among people who share a language and a culture, the difficulties multiply dangerously when it moves across borders.

What a Wall Street trader might define as moderately risky may seem downright insane to a Japanese retail broker; what an oil pipeline engineer in Brazil might characterise as gung-ho may appear overcautious to his revenue-chasing chief executive in London....The greatest pitfalls in managing risk across borders, he says, emerge from assuming too much. When dealing with fellow English speakers, it is easy to imagine that a shared language means shared assumptions - that the English, Americans and Australians think the same thing because they are using the same words.... Tips for managing risk across borders

Context is more important than language. Understand what matters most in the markets where you are doing business. Is it the law, logic or maintaining relationships?

Every word comes with its own "metadata" in different cultures. Be as specific as you can and never assume you have been properly understood without checking for potential misunderstandings.
cultural_assumptions  risks  risk-management  Communicating_&_Connecting  globalization  organizational_culture  transactions  national_identity  Philip_Delves_Broughton  translations  assumptions  misinterpretations  contextual  metadata  specificity  crossborder  cross-cultural  misunderstandings  interpretation  conversations  risk-assessment  words  compounded  risk-perception  multiplicative 
september 2012 by jerryking
Data as a Renewable Commodity and Profit Center — Jason Kolb dot Com
June 6, 2011 | Jason Kolb.com | By Jason Kolb.

One of the reasons I love data is because there’s so much potential for mining real value from it, especially when you combine it with other, new data sources. In fact it acts a lot like a traditional commodity such as copper or wool in that someone produces it, and then someone else buys the raw material and makes something new from it. It’s unique from traditional commodities, however, in that it doesn’t get used up at all when it’s used to create something new–this makes it particularly interesting from an economic point of view.

In addition, anyone can make it, it doesn’t get used up, and the industry of using data to create new and valuable things is still so young and ripe for profit-making. In fact I think it’s one of the areas that America needs to focus on if its economy is to recover because for the most part it’s still virgin territory and it’s going to create a lot of economic value. What I really don’t want to see is foreign companies being the first to capitalize on the data as that would suck most of the value out of our economy, just what we don’t need right now.
data  commercialization  renewable  commodities  metadata  Factual  Infochimps  data_scientists  information_sources 
june 2012 by jerryking
The Triumph of the Humanities - NYTimes.com
June 13, 2011,By STANLEY FISH.

There is now a (relatively) new discipline in which this breaking down
of time into spatial units that are read vertically rather than
horizontally is the obligatory gesture. It calls itself GeoHumanities
and its project is nicely encapsulated in the title of one of the essays
in a collection that officially announces the emergence of a field of
study. The collection is called “GeoHumanities: Art, History, Text at
the Edge of Place”; the essay (by Edward L. Ayers, an historian and
president of the University of Richmond) is entitled “Mapping Time.”

Ayers’s project is to map the changes that followed upon the
emancipation of the slaves after the Civil War. He and his colleagues
begin with a simple map and then they locate populations on the
landscape and “put down one layer after another: of race, of wealth, of
literacy, of water courses, of roads, of railways, of soil type, of
voting patterns, of social structure.”
Stanley_Fish  humanities  digital_humanities  geography  geohumanities  New_York  reservoirs  mapping  books  Civil_War  Emancipation  African-Americans  demographic_changes  metaphysical  metadata  overlay_networks 
june 2011 by jerryking
Scraping, cleaning, and selling big data
11 May 2011 | O'Reilly Radar | by Audrey Watters.
What are some of the challenges of acquiring data through scraping?
Flip Kromer: There are several problems with the scale and the metadata,
as well as historical complications.

Scale — It's obvious that terabytes of data will cause problems, but
so (on most filesystems) will having tens of millions of files in the
same directory tree.
Metadata — It's a chicken-and-egg problem. Since few programs can
draw on rich metadata, it's not much use annotating it. But since so few
datasets are annotated, it's not worth writing support into your
applications. We have an internal data-description language that we plan
to open source as it matures.
Historical complications — Statisticians like SPSS files. Semantic
web advocates like RDF/XML. Wall Street quants like Mathematica exports.
There is no One True Format. Lifting each out of its source domain is
time consuming.
massive_data_sets  data  analytics  data_mining  databases  digital_economy  chicken-and-egg  data_quality  metadata 
may 2011 by jerryking
Seth's Blog: Information about information
Posted by Seth Godin on July 15, 2010.

information about information. That's what Facebook and Google and Bloomberg do for a living. They create a meta-layer, a world of information about the information itself.

And why is this so valuable? Because it compounds. A tiny head start in access to this information gives you a huge advantage in the stock market. Or in marketing. Or in fundraising.

Many people and organizations are contributing to this mass of data, but few are taking advantage of the opportunity to collate it and present it to people who desperately need it. Think about how much needs to be sorted, compared, updated and presented to people who want to choose or learn or trade on it.

The race to deliver this essential scalable asset isn't over, it's just beginning.
information_flows  Information_Rules  Seth_Godin  data_driven  competingonanalytics  overlay_networks  sorting  metadata  slight_edge  compounded  inequality_of_information  multiplicative  cumulative 
july 2010 by jerryking
Needle in a haystack
Feb 27, 2010 | The Economist. Vol. 394, Iss. 8671; pg.
15.|Anonymous. AS DATA become more abundant, the main problem is no
longer finding the information as such but laying one's hands on the
relevant bits easily and quickly. What is needed is information about
information. Librarians and computer scientists call it metadata.
ProQuest  folksonomy  tagging  metadata  haystacks  commoditization_of_information  relevance 
march 2010 by jerryking
Stuck in traffic? Phone may soon help you escape - The Globe and Mail
Monday, Jan. 15, 2007 | Globe & Mail pg. A12 | by JEFF
GRAY. "In the surprisingly near future, your cellphone may be able to
warn you about a traffic jam ahead, predict precisely how long your
commute home will take, or even recommend an alternative route."
computers essentially take a look at the torrent of data this "pinging"
pours in, using a "triangulation" process based on the time-delay
between pings. Its system figures out which cellphones are moving, where
they are, and how fast they are going. The data are then streamed into a
traffic map and produce precise information on speeds and estimated
travel times not just on major expressways, but on every single road in
cellphone range.
Jeff_Gray  mobile_phones  triangulation  privacy  congestion  competingonanalytics  data_mining  massive_data_sets  location_based_services  metadata  traffic_congestion 
october 2009 by jerryking
DaaS: The New Information Goldmine - WSJ.com
AUGUST 19, 2009 | Wall Street Journal | by DYAN MACHAN. Data
as a Service, or its diminutive, DaaS. It rhymes with SaaS, its
better-known cousin that stands for Software as a Service. SaaS is the
catchall name for on-demand software applications like those on an
iPhone. DaaS, in contrast, recognizes that software is becoming a
commodity; it's data mixed with software that's king.
SaaS  Freshbooks  data  DaaS  data_driven  metadata 
august 2009 by jerryking
The Medium - Photo Negative - Google Misses an Opportunity With Its Life Magazine Archive - NYTimes.com
February 27, 2009 NYT Magazine article VIRGINIA HEFFERNAN who
is mystified by Google’s recent decision to essentially dump its
priceless trove of photos from Life magazine — some 10 million images
from Life’s holdings, most of them never published — into an online
crate.
Google has failed to recognize that it can’t publish content under its
imprint without also creating content of some kind: smart, reported
captions; new and good-looking slide-show software; interstitial
material that connects disparate photos; robust thematic and topical
organization.
Google  photography  curation  content  Life_Magazine  storytelling  interstitial  overlay_networks  jazmin_isaacs  metadata  missed_opportunities  contextual  sorting  creating_valuable_content 
march 2009 by jerryking

Copy this bookmark:





to read