algorithms everywhere
Post on 29-Nov-2015
10 Views
Preview:
DESCRIPTION
TRANSCRIPT
Created by Michael Reilly December 2011
1
Algos everywhere: new policies needed for the invisible processes shaping
our lives?
Summary
Algorithms can be used to develop scaleable processes that produce highly valuable
outputs. There are a growing number and variety of real-world algo applications
including computer-based trading in financial markets, logistic optimisation,
cryptography, retail market intelligence, preventative medicine, online dating and job
matching, large-scale surveillance and investigating paintings with disputed
attribution. Some experts argue that algorithms are a general purpose technology
that will revolutionise our lives. But algorithms are rarely objective and often there is
a lack of transparency regarding their use. Policies on the use of algorithms have not
evolved at the same rate as their development and negative externalities are far from
negligible.
Discussion
An algorithm is a problem-solving process that produces an output from given inputs.
With the invention of enabling general purpose technologies such as the computer
and the internet, algorithms have become pervasive in human life. In business,
algorithms can confer considerable comparative advantages by supporting
economies of scale for very large inputs. For example, large-scale logistical
problems have proved tractable to algorithms. United Parcel Service (UPS) uses
algorithms to optimise the huge number of possible driver routes that are used to
deliver millions of packages each day1. Algorithms are used to improve the efficiency
of air traffic control. Algorithms are also being used to transform the deluge of data
produced by modern retail transactions into highly valuable market intelligence.
Researchers from the Hewlett Packard Labs in Palo Alto are patenting an algorithm
that analyses the sentiments expressed in tweets about films. This algorithm
predicted the box office returns of a sample of films in their opening weekend with 98
per cent accuracy2. In the financial sector, algorithms have supplanted human
traders in equity and FX markets because they can make decisions on whether to
buy and sell financial instruments at the scale of microseconds. Algorithms such as
CrowdForge coordinate human workers through piecework websites like Mechanical
Turk3.
For many years there have been important national security applications for
algorithms that encode and decode information – cryptography is used extensively in
both the private and the public sector. Recent novel applications of algorithms
1 Economist. 2011. Business by numbers. http://www.economist.com/node/9795140.
2 Harkin, J. 2011. How ‘infodemiology’ is quantifying you. Wired.
http://www.wired.co.uk/magazine/archive/2011/06/ideas-bank/james-harkin-infodemiology. 3 Economist. 2011. Return of the human computers. http://www.economist.com/node/21540393.
Created by Michael Reilly December 2011
2
include interpreting fMRI scans so as to quantify the risks of psychosis4, producing
automated sports journalism5, and investigating the attribution and provenance of
paintings6. The Heritage Provider Network has offered a $3m prize to the algorithm
that can best predict when people are likely to be sent to hospital and, therefore,
help to create a better model for preventive medicine7. As much as 40% of single
people in the US are using algorithms to find a partner; and a survey in 2007 found
that 2 per cent of all marriages in the US were facilitated by the EHarmony online
dating website alone8.
The growing number of applications for algorithms has led some experts to argue
that algorithms are a general purpose technology that will revolutionise our lives9.
Implications
Policies on the use of algorithms have not evolved at the same rate as their
development. There have been recent instances of negative externalities associated
with the use of algorithms, which may be related to the lack of transparency
surrounding their application. Algorithms are rarely objective and often have a hidden
agenda.
High-frequency trading in financial markets using algorithms may improve important
outcomes such as liquidity and price efficiency but in conditions of market stress it
can also induce mayhem10. On May 6th 2010 there was a “Flash Crash” in US equity
markets that eliminated approximately $800bn of value in 5 minutes only to regain
almost all of the losses within 30 minutes. This event eroded confidence in stock
markets and was followed by several months of outflows from retail mutual funds in
the US. Algorithms may be creating ‘filter bubbles’ on the World Wide Web that
constrain our access to information with newsfeeds and recommendations that
reinforce rather than challenge our views; this form of ‘algorithmic paternalism’ could
lead to a more homogenous and less resilient society11. The internet is an intrinsic
4 Mourao-Miranda J, Reinders AA, Rocha-Rego V, Lappin J, Rondina J, Morgan C, Morgan KD,
Fearon P, Jones PB, Doody GA, Murray RM, Kapur S, Dazzan P. 2011. Individualized prediction of illness course at the first psychotic episode: a support vector machine MRI study. Psychological Medicine. 5 Lohr, S. 2011. In Case You Wondered, a Real Human Wrote This Column. New York Times.
http://www.nytimes.com/2011/09/11/business/computer-generated-articles-are-gaining-traction.html?_r=1. 6 Ju, A. 2011. Engineering professor uses tools of his trade to count Van Gogh canvas threads.
Cornell University Chronicle online. http://www.news.cornell.edu/stories/March11/JohnsonVanGogh.html. 7 Valentino-Devries, J. 2011. May the best algorithm win. Wall Street Journal.
http://online.wsj.com/article/SB10001424052748704662604576202392747278936.html#ixzz1gRPE2UGU. 8 Bialik, C. 2009. How many marriages started online? Wall Street Journal.
http://blogs.wsj.com/numbersguy/how-many-marriages-started-online-764/. 9 Chazelle, B. 2006. The Algorithm: Idiom of Modern Science.
http://www.cs.princeton.edu/~chazelle/pubs/algorithm.html. 10
Foresight. 2011. The Future of Computer Trading in Financial Markets Working Paper. 11
O’Callaghan, T. 2011. Breaking out of the internet filter bubble. New Scientist. http://www.newscientist.com/blogs/culturelab/2011/06/why-facebook-have-an-important-button.html.
Created by Michael Reilly December 2011
3
part of the economy and there are financial gains to be made from influencing search
engine results through both legal and illegal means. The practice of subverting
Google’s search engine to return a specific result, or ‘Googlebombing’, has persisted
since its inception.
Algorithms may support economies of scale for very large inputs but this can result in
a trade-off with the quality of its outputs. In the words of Albert Einstein “not
everything that can be counted counts, and not everything that counts can be
counted”. This notwithstanding, more sophisticated algorithms are being developed
that can respond to feedback and adapt like living creatures to their environment12.
Algorithms are the centre of a continuing debate on security versus privacy. In
Shenzhen, a surveillance system called ‘Golden Shield’ uses algorithms to fuse the
data provided by CCTV footage, World Wide Web trails, mobile telephone use and
GPS so as to track social unrest13. Much of the technology for this system was
actually provided by Western companies. Facial recognition software was used to
identify participants in the London riots of 2011; and new algorithms are being
developed to recognise specific types of criminal behaviour14.
Algorithms have the potential to be a driver of economic growth. The success of
Google was founded on the application of a ‘PageRank’ algorithm developed by
Larry Page and Sergey Brin at Stanford University in the 1990s, which produced
relevant search results from very large numbers of World Wide Web pages. Google
currently has a market capitalisation of over $200bn. The value of the still nascent
global mobile telephone apps market in 2010 was around $7bn and may increase to
$25bn by 201515. In a globalised world facing rising energy costs and CO2
emissions, algorithms that efficiently solve problems for a given set of inputs could
help to mitigate climate change and ameliorate the impacts on developing world
economic growth. One researcher “imagines a future in which algorithms co-ordinate
an army of human workers, physical sensors and conventional computers”16.
Using algorithms to match people to jobs has the potential to improve the flexibility of
labour markets. Daniel Kahnemann, one of the pioneers of behavioural psychology,
argues that in many instances, including job interviews, algorithms can mitigate
human cognitive biases17. On the other hand, if these algorithms are subjectively
configured they could damage the life chances of individuals and exacerbate
widening social inequality through non-linear feedback effects.
12
Economist. 2007. Of greed and ants. http://www.economist.com/node/9796508. 13
Klein, N. 2008. China’s all-seeing eye. Rolling Stone. http://www.naomiklein.org/articles/2008/05/chinas-all-seeing-eye. 14
Dillow, C. 2011. Smart CCTV System Would Use Algorithm to Zero in on Crime-Like Behaviour. Popular Science. http://www.popsci.com/technology/article/2011-08/new-cctv-system-would-use-behavior-recognition-zero-crimes. 15
Wauters, R. 2011. Report: Mobile App Market Will Be Worth $25 Billion By 2015 – Apple's Share: 20%. Tech Crunch. 16
Economist. 2011. Return of the human computers. http://www.economist.com/node/21540393. 17
Kahnemann, D. 2001. Thinking fast and slow. Farrar, Strauss and Giroux: New York.
Created by Michael Reilly December 2011
4
Early indicators
Flash crashes in financial markets
Internet filter bubbles
Rate of growth of mobile phone apps market
Algorithm competitions
China’s ‘Golden Shield’ system of surveillance
Increase in online dating
Increase in online job matching
Drivers and Inhibitors
Drivers: Increased trade, Data deluge, Consumerism, Growing internet use, Health
costs, Open innovation, Climate Change, Emerging middle class in the developing
world, Computing power, Security, Public disorder, Distrust of experts
Inhibitors: Public mistrust, Activism, Data protection, Deglobalisation, Human
cognitive biases, Political costs of unemployment
Parallels and Precedents
Parallels: Data-intensive science, Evolutionary processes
Precedents: Google, Mobile telephone apps market, Logistic optimisation,
Cryptography
top related