Nicholas Christakis muestra como las redes sociales ayudan a predecir epidemias
Transcripción
Durante los últimos 10 años he tratado de dar con la forma y la razón por la que los seres humanos se congregan en redes sociales. Y el tipo de red social de la que hablo no es la reciente variedad virtual sino más bien del tipo de redes sociales en las que los humanos nos hemos congregado durante cientos y miles de años desde que aparecimos en la sabana africana. Así, entablo amistades y relaciones laborales, y relaciones fraternales y de parentesco con otra gente, quienes a su vez tienen relaciones con otras personas. Y esto se extiende indefinidamente en la distancia. Y se tiene una red que se parece a esto. Cada punto es una persona. Cada línea entre puntos es una relación entre dos personas... distintos tipos de relaciones. Así se obtiene esta suerte de tejido de humanidad en el que todos estamos inmersos.
Con mi colega James Fowler hemos estado estudiando durante bastante tiempo las reglas matemáticas, sociales, biológicas y psicológicas que gobiernan la forma de ensamblaje de estas redes y las reglas similares que gobiernan la forma en que operan, en que afectan nuestras vidas. Y hace poco nos preguntábamos si sería posible sacar ventaja de este conocimiento para encontrar maneras de mejorar el mundo, de hacer algo mejor, para solucionar las cosas y no sólo para entenderlas. Uno de los primeros temas que pensamos abordar era la forma de predecir epidemias.
Y la técnica actual para predecir epidemias -en el CDC o algún otro organismo nacional- es sentarse en el lugar que uno esté y recolectar datos de médicos y laboratorios en el terreno que informen de la prevalencia o la incidencia de ciertas condiciones. Pacientes así, así y así han sido diagnosticados [por aquí] u otros pacientes lo han sido [por allí] y todos estos datos van a un repositorio central con cierta demora. Y si todo va bien, en una o dos semanas, uno sabrá en qué estado estaba hoy la epidemia. En realidad hace cosa de un año se hizo conocida esta noción de tendencia gripal en Google, en relación a la gripe, según la cual mirando el comportamiento de búsqueda hoy podíamos saber de la gripe... el estado actual de la epidemia, la prevalecencia actual de la epidemia.
Pero lo que hoy quiero mostrarles es un medio por el cual podríamos tener no sólo alertas rápidas de una epidemia sino también detección temprana de la misma. Y, de hecho, esta idea puede usarse no sólo para predecir epidemias de gérmenes sino también para predecir epidemias de todo tipo. Por ejemplo, cualquier cosa que se propague por contagio social puede ser comprendida de este modo, desde las ideas de la izquierda como el patriotismo, el altruismo o la religión hasta prácticas como las dietas o la compra de libros bebidas, cascos de bicicleta [y] otras medidas de seguridad; o productos que la gente podría adquirir, la compra de artículos electrónicos, cualquier cosa en la que haya propagación interpersonal. Una especie de difusión de la innovación podría ser entendida y predicha con el mecanismo que ahora voy a mostrarles.
Así, como todos probablemente saben, la manera clásica de pensar en esto es la "difusión de la innovación" o "curva de adopción". Aquí en el eje Y tenemos el porcentaje de personas afectadas y en el eje X tenemos el tiempo. Al principio no hay demasiadas personas afectadas, y se tiene esta curva sigmoidea clásica o curva en forma de S. La razón de esta forma es que muy al principio digamos que una o dos personas están afectadas o infectadas y luego ellos afectan, o infectan, a dos personas que a su vez afectan a 4, 8, 16, etc., y se obtiene la fase de crecimiento de la epidemia en la curva. Y, finalmente, se satura la población. Hay cada vez menos personas que todavía pueden ser infectadas y entonces se tiene la meseta de la curva, y se obtiene esta curva sigmoidea clásica. Y esto vale para gérmenes, ideas, adopción de productos, comportamientos y similares. Pero las cosas no se difunden aleatoriamente en las poblaciones humanas. Se difunden en redes. Porque, como dije, vivimos nuestras vidas en redes y estas redes tienen un tipo particular de estructura.
Ahora, si vemos una red como ésta... Ésta tiene 105 personas. Y las líneas representan... los puntos son las personas y las líneas las relaciones de amistad. Puede verse que las personas ocupan distintas ubicaciones en la red. Y hay distintos tipos de relaciones entre las personas. Pueden darse relaciones de amistad, relaciones fraternales, relaciones maritales, relaciones laborales, relaciones vecinales, etc. Y distintos tipos de cosas se difunden por diferentes tipos de lazos. Por ejemplo, las enfermedades de transmisión sexual se esparcirán por los vínculos sexuales. O, por ejemplo, el hábito de fumar podría ser influencia de los amigos. O el altruismo y las donaciones caritativas podrían estar influenciados por los compañeros de trabajo, o por los vecinos. Pero no todas las ubicaciones de la red son iguales.
Así, si miran esto van a captar de inmediato que diferentes personas tienen distinta cantidad de conexiones. Algunas personas tienen 1 conexión, algunas tienen 2, algunas tienen 6, algunas tienen 10. Y esto se llama el "grado" de un nodo o la cantidad de conexiones que tiene un nodo. Pero hay algo más. Si uno mira los nodos A y B, ambos tienen 6 conexiones. Pero si uno mira esta imagen [de la red] a vista de pájaro, puede apreciar que hay algo muy diferente entre los nodos A y B. Déjenme preguntarles esto, puedo fomentar esta intuición haciendo una pregunta: ¿Quién les gustaría ser si un germen se esparciera por la red, A o B? (Audiencia: B) Nicholas Christakis: B, obviamente. B está ubicado al borde de la red. Ahora, ¿quién les gustaría ser si se propagara por la red un chisme jugoso? A. Y uno tiene una apreciación inmediata de que va a ser más probable que A dé primero con lo que se está propagando en virtud de su ubicación estructural dentro de la red. A, de hecho, es más central y esto se puede formalizar matemáticamente. Por lo tanto, si quisiéramos seguir algo que se está propagando por una red, lo ideal sería poner sensores en los individuos del centro de la red, incluyendo el nodo A, y monitorear a esas personas que están justo en el centro de la red, para, de algún modo, detectar en forma temprana lo que sea que se esté difundiendo por la red.
Así, si los viéramos contraer un germen o una información uno sabría, muy pronto, que todos están por contraer el germen o por enterarse de esa información. Y esto sería mucho mejor que monitorear a 6 personas elegidas al azar sin hacer referencia a la estructura de la población. Y, de hecho, si uno pudiera hacer eso lo que vería sería algo como esto. De nuevo, en el panel de la izquierda tenemos la curva de adopción en forma de S. En la línea roja punteada mostramos cómo sería la adopción en las personas elegidas al azar y en la línea de la izquierda, desplazada a la izquierda, mostramos cómo sería la adopción en los individuos del centro de la red. En el eje Y están las instancias acumulativas de contagio y en el eje X está el tiempo. Y a la derecha, mostramos los mismos datos, pero aquí con incidencia diaria. Y lo que aquí mostramos, como en este caso, es que hay pocas personas afectadas, cada vez más y más hasta llegar aquí, y aquí es el pico de la epidemia. Pero desplazado a la izquierda se ve lo que ocurre con los individuos del centro. Y esta diferencia en tiempo entre los dos es la detección temprana, la alerta temprana que se dispara sobre una epidemia inminente en la población humana.
El problema, sin embargo, es que el mapeo de redes sociales humanas no siempre es posible. Puede que sea caro, [muy difícil], antiético, o, francamente, imposible de realizar. Entonces: ¿cómo podemos averiguar cuáles son las personas centrales de una red sin, de hecho, mapear la red? Y surgió la idea de explotar un hecho muy antiguo o un hecho conocido de las redes sociales que dice así: ¿sabías que tus amigos tienen más amigos que tú? Tus amigos tienen más amigos que tú. Y esto se conoce como la paradoja de la amistad. Imaginen una persona muy popular en la red social -como un anfitrión de una fiesta con cientos de amigos- y un misántropo que tiene un solo amigo; si uno toma una persona al azar, es mucho más probable que conozca al anfitrión. Y si señala al anfitrión como amigo, ese anfitrión tiene cientos de amigos, por lo tanto tiene más amigos que uno. Y esto, en esencia, es lo que se conoce como la paradoja de la amistad. Los amigos de las personas elegidas al azar tienen más alto grado y están más al centro que la propia gente elegida al azar.
Y uno puede hacerse una idea intuitiva de esto si piensa en la gente del perímetro de la red. Si uno elige a esta persona el único amigo que tendrá para elegir es esta persona, quien, por construcción, debe tener al menos dos o, normalmente, más amigos. Y eso sucede en cada nodo periférico. De hecho, sucede en toda la red conforme uno se desplaza, cualquiera que elijamos, cuando nomine alguien al azar... cuando una persona al azar nomina a un amigo uno se mueve más cerca del centro de la red. Así, pensamos en explotar esta idea para estudiar si podíamos predecir fenómenos dentro de las redes. Porque ahora, con esta idea, podemos tomar una muestra aleatoria de gente, hacer que elijan a sus amigos, y esos amigos estarán más al centro, y podríamos hacer eso sin tener que mapear la red.
Hemos probado esta idea con un brote de la gripe H1N1 en la universidad de Harvard en el otoño y el invierno de 2009, hace apenas unos meses. Tomamos 1.300 estudiantes seleccionados al azar, hicimos que elijan a sus amigos y seguimos a los estudiantes elegidos al azar y a sus amigos diariamente para ver si tenían o no la epidemia de gripe. Y lo hicimos pasivamente observando si habían ido a los servicios de salud universitarios. Les pedimos también que nos envíen un correo un par de veces por semana. Y sucedió exactamente lo que predijimos. El grupo aleatorio está en la línea roja. La epidemia en el grupo de amigos se desplazó a la izquierda, por aquí. Y la diferencia entre los dos es de 16 días. Monitoreando el grupo de amigos pudimos tener una alerta 16 días antes de una epidemia inminente en esta población humana.
Ahora, además de eso, si uno fuese un analista que trata de estudiar una epidemia o de predecir la adopción de un producto, por ejemplo, lo que podría hacer es tomar una muestra aleatoria de la población, pedirle que elijan a sus amigos y seguir a los amigos, y seguir tanto a los aleatorios como a los amigos. Entre los amigos, la primera evidencia de un salto sobre cero en la adopción de innovación, por ejemplo, sería la evidencia de una epidemia inminente. O uno podría ver la primera vez que divergen las dos curvas, como se muestra a la izquierda. ¿Cuándo los aleatorios... cuándo despegaron los amigos y dejaron a los aleatorios y su curva empezó a desplazarse? Y eso, como indica la línea blanca, se produjo 46 días antes del pico de la epidemia. Así que esta sería una técnica mediante la cual se podría alertar más de un mes y medio antes la epidemia de gripe en una población en particular.
Debo decir que la antelación con la que puede conocerse una noticia depende de una serie de factores. Podría depender de la naturaleza del patógeno, distintos patógenos, usando esta técnica, se obtienen distintas alertas, u otros fenómenos que se están extendiendo o, francamente, de la estructura de la red humana. En nuestro caso, aunque no era necesario, pudimos, de hecho, mapear la red de estudiantes.
Este es un mapeo de 714 estudiantes y sus vínculos de amistad. Y en un minuto voy a poner este mapeo en movimiento. Vamos a tomar cortes diarios de la red durante 120 días. Los puntos rojos van a ser casos de gripe y los puntos amarillos van a ser amigos de las personas con gripe. Y el tamaño de los puntos va a ser proporcional a la cantidad de amigos con gripe. Así, puntos más grandes significan más amigos con gripe. Y si miran esta imagen, aquí estamos en el 13 de septiembre, van a ver algunos casos iluminados. Van a ver una especie de florecimiento de la gripe en el medio. Aquí estamos el 19 de octubre. La pendiente de la curva de la epidemia se está acercando ahora, en noviembre. Bang, bang, bang, bang, van a ver un gran florecimiento en el medio, y luego van a ver una especie de nivelación, cada vez menos casos hasta fines de diciembre. Y este tipo de visualización puede mostrar que las epidemias como éstas echan raíces y afectan primero a los individuos del centro antes de afectar a otros.
Ahora, como he estado sugiriendo, este método no se limita a los gérmenes, sino, en realidad, a cualquier cosa que se propague en la población. La información se propaga en la población. Las normas se propagan en la población. Los comportamientos pueden propagarse en la población. Y comportamiento puede significar comportamiento criminal o electoral, o del cuidado de la salud como el tabaco o las vacunas, o la adopción de productos u otro tipo de comportamiento relacionado con la influencia interpersonal. Si soy capaz de hacer algo que afecta a los demás a mi alrededor, esta técnica puede proporcionar una alerta o detección temprana sobre la adopción en la población. La clave es que, para que funcione, tiene que haber influencia interpersonal. No puede deberse a un mecanismo de difusión que afecte a todos por igual.
Ahora, los mismos conocimientos de las redes pueden ser explotados también de otras maneras por ejemplo, para seleccionar personas específicas para intervenciones. Muchos de Uds. están familiarizados probablemente con la noción de inmunidad de grupo. Así, si tenemos una población de mil personas y queremos hacer que la población sea inmune a un patógeno no tenemos que inmunizar a todos. Si inmunizamos a 960 de ellos es como si hubiéramos inmunizado al 100% de ellos. Porque incluso si una o dos de las personas no inmunes se infectan, no hay nadie a quien puedan infectar. Están rodeados de personas inmunizadas. Así que el 96% es tan bueno como el 100%. Bueno, algunos científicos han estimado qué pasaría si se toma una muestra aleatoria del 30% y de estas 1.000 personas se inmuniza a 300. ¿Se obtendría alguna inmunidad a nivel poblacional? Y la respuesta es no. Pero si uno toma este 30%, estas 300 personas, y hacemos que elijan a sus amigos y tomamos la misma cantidad de dosis de vacunas y vacunamos a los amigos de los 300, los 300 amigos, se obtiene el mismo nivel de inmunidad de grupo que si vacunáramos al 96% de la población con mucha mayor eficiencia y una restricción presupuestaria estricta.
Y pueden usarse ideas similares, por ejemplo, para enfocar la distribucion de cosas como mosquiteros en el mundo en desarrollo. Si pudiésemos identificar la estructura de las redes en los pueblos podríamos elegir a quién darle las intervenciones para fomentar este tipo de propagación. O bien, francamente, para publicitar todo tipo de productos. Si pudiéramos entender cómo seleccionar, eso podría afectar la eficiencia de lo que estamos tratando de lograr. Y, de hecho, podemos usar datos de todo tipo de fuentes hoy en día [para hacerlo].
Este es un mapeo de 8 millones de usuarios de teléfono en un país europeo. Cada punto es una persona, y cada línea representa un volumen de llamadas entre personas. Y podemos usar estos datos obtenidos de manera pasiva para mapear estos países enteros y comprender dónde se ubica cada quien en la red. Sin tener que interrogarlos en absoluto podemos obtener este tipo de conocimiento estructural. Y otras fuentes de información, que Uds. sin duda conocen, están disponibles a partir de las interacciones de correo electrónico, interacciones en línea, redes sociales virtuales, etc. Y, de hecho, estamos en la era de lo que llamaría esfuerzos de recolección de datos "masivo-pasivos". Hay todo tipo de maneras de recolectar datos en forma masiva para crear redes de sensores para seguir a la población y comprender lo que sucede en la población e intervenir en la población para mejor. Porque estas nuevas tecnologías nos dicen no sólo quién habla con quién sino dónde está cada uno y lo que están pensando en base a lo que están subiendo a internet, y lo que están consumiendo en base a sus compras. Y todos estos datos administrativos pueden juntarse y ser procesados para comprender el comportamiento humano en modos nunca antes posibles.
Por ejemplo: podríamos usar las compras de combustible de transportistas. Los transportistas hacen lo suyo y compran combustible. Vemos una suba en la compra de combustible de los transportistas y sabemos que una recesión está por terminar. O podemos analizar la velocidad a la que se mueve la gente con sus celulares en la autopista y la compañía telefónica puede ver, conforme la velocidad disminuye, que hay un atasco de tráfico. Y se puede enviar esa información a los clientes, pero sólo a los clientes que están en esa misma autopista ubicados detrás del atasco. O podemos monitorear diagnósticos médicos, de forma pasiva, y ver la difusión de innovación en productos farmacéuticos en las redes de médicos. O, de vuelta, podemos seguir los hábitos de compra de la gente y observar cómo estos tipos de fenómenos pueden difundirse en las poblaciones humanas.
Y creo que hay tres maneras en que pueden usarse estos datos masivo-pasivos. Una es totalmente pasiva como acabo de describir. Por ejemplo, el caso de los transportistas en el que no se interviene en la población de ningún modo. Otro es casi activo, como el ejemplo que di de la gripe, en el que pedimos a la gente que elija a sus amigos, y luego controlamos pasivamente a sus amigos: ¿tienen la gripe o no? Y luego la advertencia. Y otro ejemplo sería si uno es una compañía telefónica, averigua quiénes son el centro de la red, y le pregunta a esa gente: "¿Podrías mandarnos un sms con tu temperatura todos los días? Y uno junta ingentes cantidades de información de la temperatura de la gente pero de los individuos del centro. Y se es capaz, a gran escala, de monitorear una epidemia inminente con una participación mínima de la gente. O podría ser algo completamente activo, sé que los próximos oradores también hablarán de esto hoy, donde la gente participará globalmente en wikis o fotografiando, o siguiendo elecciones, y subiendo información de modo que pueda consolidarse para comprender los procesos sociales y los fenómenos sociales.
De hecho, la disponibilidad de estos datos, creo, anuncian una nueva era de lo que tanto yo como otros daríamos en llamar "ciencias sociales de cómputo". Es como cuando Galileo inventó -o no inventó- utilizó un telescopio y pudo ver el firmamento de otra manera; o cuando Leeuwenhoek conoció el microscopio -o en realidad lo inventó- y pudo ver la biología de manera nueva. Pero ahora tenemos acceso a estos datos que nos permiten entender los procesos sociales y los fenómenos sociales de una forma totalmente nueva que nunca antes fue posible. Y con esa ciencia podemos entender exactamente cómo el todo viene a ser más grande que la suma de las partes. Y, en realidad, podemos usar estos conocimientos para mejorar la sociedad y el bienestar del hombre.
Gracias.
TED
lunes, 20 de mayo de 2013
jueves, 9 de mayo de 2013
Como las redes sociales mantienen el desempleo de los afroamericanos en USA
How Social Networks Drive Black Unemployment
By NANCY DITOMASO
It’s easy to believe the worst is over in the economic downturn. But for African-Americans, the pain continues — over 13 percent of black workers are unemployed, nearly twice the national average. And that’s not a new development: regardless of the economy, job prospects for African-Americans have long been significantly worse than for the country as a whole.
The most obvious explanation for this entrenched disparity is racial discrimination. But in my research I have found a somewhat different culprit: favoritism. Getting an inside edge by using help from family and friends is a powerful, hidden force driving inequality in the United States.
Such favoritism has a strong racial component. Through such seemingly innocuous networking, white Americans tend to help other whites, because social resources are concentrated among whites. If African-Americans are not part of the same networks, they will have a harder time finding decent jobs.
The mechanism that reproduces inequality, in other words, may be inclusion more than exclusion. And while exclusion or discrimination is illegal, inclusion or favoritism is not — meaning it can be more insidious and largely immune to legal challenges.
Favoritism is almost universal in today’s job market. In interviews with hundreds of people on this topic, I found that all but a handful used the help of family and friends to find 70 percent of the jobs they held over their lifetimes; they all used personal networks and insider information if it was available to them.
In this context of widespread networking, the idea that there is a job “market” based solely on skills, qualifications and merit is false. Whenever possible, Americans seeking jobs try to avoid market competition: they look for unequal rather than equal opportunity. In fact, the last thing job seekers want to face is equal opportunity; they want an advantage. They want to find ways to cut in line and get ahead.
You don’t usually need a strong social network to land a low-wage job at a fast-food restaurant or retail store. But trying to land a coveted position that offers a good salary and benefits is a different story. To gain an edge, job seekers actively work connections with friends and family members in pursuit of these opportunities.
Help is not given to just anyone, nor is it available from everyone. Inequality reproduces itself because help is typically reserved for people who are “like me”: the people who live in my neighborhood, those who attend my church or school or those with whom I have worked in the past. It is only natural that when there are jobs to be had, people who know about them will tell the people who are close to them, those with whom they identify, and those who at some point can reciprocate the favor.
Because we still live largely segregated lives, such networking fosters categorical inequality: whites help other whites, especially when unemployment is high. Although people from every background may try to help their own, whites are more likely to hold the sorts of jobs that are protected from market competition, that pay a living wage and that have the potential to teach skills and allow for job training and advancement. So, just as opportunities are unequally distributed, they are also unequally redistributed.
All of this may make sense intuitively, but most people are unaware of the way racial ties affect their job prospects.
When I asked my interviewees what most contributed to their level of career success, they usually discussed how hard they had worked and how uncertain were the outcomes — not the help they had received throughout their lives to gain most of their jobs. In fact, only 14 percent mentioned that they had received help of any kind from others. Seeing contemporary labor-market politics through the lens of favoritism, rather than discrimination alone, is revealing. It explains, for example, why even though the majority of all Americans, including whites, support civil rights in principle, there is widespread opposition on the part of many whites to affirmative action policies — despite complaints about “reverse discrimination,” my research demonstrated that the real complaint is that affirmative action undermines long-established patterns of favoritism.
The interviewees in my study who were most angry about affirmative action were those who had relatively fewer marketable skills — and were therefore most dependent on getting an inside edge for the best jobs. Whites who felt entitled to these positions believed that affirmative action was unfair because it blocked their own privileged access.
But interviewees’ feelings about such policies betrayed the reality of their experience of them. I found these attitudes evident among my interviewees — even though, among the 1,463 jobs they discussed with me, there were only two cases in which someone might have been passed over for a job because of affirmative action policies benefiting African-Americans. These data are consistent with other research on affirmative action.
There’s no question that discrimination is still a problem in the American economy. But whites helping other whites is not the same as discrimination, and it is not illegal. Yet it may have a powerful effect on the access that African-Americans and other minorities have to good jobs, or even to the job market itself.
Nancy DiTomaso, the vice dean for faculty and research and a professor of management and global business at Rutgers Business School, is the author of “The American Non-Dilemma: Racial Inequality Without Racism.”
miércoles, 8 de mayo de 2013
Datos científicos de Facebook gracias a Wolfram
Data Science of the Facebook World
April 24, 2013
|
More than a million people have now used our Wolfram|Alpha Personal Analytics for Facebook. And as part of our latest update, in addition to collecting some anonymized statistics, we launched a Data Donor program that allows people to contribute detailed data to us for research purposes.
A few weeks ago we decided to start analyzing all this data. And I have to say that if nothing else it’s been a terrific example of the power of Mathematica and the Wolfram Language for doing data science. (It’ll also be good fodder for the Data Science course I’m starting to create.)
We’d always planned to use the data we collect to enhance our Personal Analyticssystem. But I couldn’t resist also trying to do some basic science with it.
I’ve always been interested in people and the trajectories of their lives. But I’ve never been able to combine that with my interest in science. Until now. And it’s been quite a thrill over the past few weeks to see the results we’ve been able to get. Sometimes confirming impressions I’ve had; sometimes showing things I never would have guessed. And all along reminding me of phenomena I’ve studied scientifically in A New Kind of Science.
So what does the data look like? Here are the social networks of a few Data Donors—with clusters of friends given different colors. (Anyone can find their own network using Wolfram|Alpha—or the
SocialMediaData function inMathematica.)
So a first quantitative question to ask is: How big are these networks usually? In other words, how many friends do people typically have on Facebook? Well, at least for our users, that’s easy to answer. The median is 342—and here’s a histogram showing the distribution (there’s a cutoff at 5000 because that’s the maximum number of friends for a personal Facebook page):
But how typical are our users? In most respects—so far as we can tell—they seem pretty typical. But there are definitely some differences. Like here’s the distribution of the number of friends not just for our users, but also for their friends (there’s a mathematical subtlety in deriving this that I’ll discuss later):
And what we see is that in this broader Facebook population, there are significantly more people who have almost no Facebook friends. Whether such people should be included in samples one takes is a matter of debate. But so long as one looks at appropriate comparisons, aggregates, and so on, they don’t seem to have a huge effect. (The spike at 200 friends probably has to do with Facebook’s friend recommendation system.)
So, OK. Let’s ask for example how the typical number of Facebook friends varies with a person’s age. Of course all we know are self-reported “Facebook ages”. But let’s plot how the number of friends varies with that age. The solid line is the median number of friends; successive bands show successive octiles of the distribution.
After a rapid rise, the number of friends peaks for people in their late teenage years, and then declines thereafter. Why is this? I suspect it’s partly a reflection of people’s intrinsic behavior, and partly a reflection of the fact that Facebook hasn’t yet been around very long. Assuming people don’t drop friends much once they’ve added them one might expect that the number of friends would simply grow with age. And for sufficiently young people that’s basically what we see. But there’s a limit to the growth, because there’s a limit to the number of years people have been on Facebook. And assuming that’s roughly constant across ages, what the plot suggests is that people add friends progressively more slowly with age.
But what friends do they add? Given a person of a particular age, we can for example ask what the distribution of ages of the person’s friends is. Here are some results (the jaggedness, particularly at age 70, comes from the limited data we have):
And here’s an interactive version, generated from CDF:
The first thing we see is that the ages of friends always peak at or near the age of the person themselves—which is presumably a reflection of the fact that in today’s society many friends are made in age-based classes in school or college. For younger people, the peak around the person’s age tends to be pretty sharp. For older people, the distribution gets progressively broader.
We can summarize what happens by plotting the distribution of friend ages against the age of a person (the solid line is the median age of friends):
There’s an anomaly for the youngest ages, presumably because of kids under 13 misreporting their ages. But apart from that, we see that young people tend to have friends who are remarkably close in age to themselves. The broadening as people get older is probably associated with people making non-age-related friends in their workplaces and communities. And as the array of plots above suggests, by people’s mid-40s, there start to be secondary peaks at younger ages, presumably as people’s children become teenagers, and start using Facebook.
So what else can one see about the trajectory of people’s lives? Here’s the breakdown according to reported relationship status as a function of age:
And here’s more detail, separating out fractions for males and females (“married+” means “civil union”, “separated”, “widowed”, etc. as well as “married”):
There’s some obvious goofiness at low ages with kids (slightly more often girls than boys) misreporting themselves as married. But in general the trend is clear. The rate of getting married starts going up in the early 20s—a couple of years earlier for women than for men—and decreases again in the late 30s, with about 70% of people by then being married. The fraction of people “in a relationship” peaks around age 24, and there’s a small “engaged” peak around 27. The fraction of people who report themselves as married continues to increase roughly linearly with age, gaining about 5% between age 40 and age 60—while the fraction of people who report themselves as single continues to increase for women, while decreasing for men.
I have to say that as I look at the plots above, I’m struck by their similarity to plots for physical processes like chemical reactions. It’s as if all those humans, with all the complexities of their lives, still behave in aggregate a bit like molecules—with certain “reaction rates” to enter into relationships, marry, etc.
Of course, what we’re seeing here is just for the “Facebook world”. So how does it compare to the world at large? Well, at least some of what we can measure in the Facebook world is also measured in official censuses. And so for example we can see how our results for the fraction of people married at a given age compare with results from the official US Census:
I’m amazed at how close the correspondence is. Though there are clearly some differences. Like below age 20 kids on Facebook are misreporting themselves as married. And on the older end, widows are still considering themselves married for purposes of Facebook. For people in their 20s, there’s also a small systematic difference—with people on Facebook on average getting married a couple of years later than the Census would suggest. (As one might expect, if one excludes the rural US population, the difference gets significantly smaller.)
Talking of the Census, we can ask in general how our Facebook population compares to the US population. And for example, we find, not surprisingly, that our Facebook population is heavily weighted toward younger people:
OK. So we saw above how the typical number of friends a person has depends on age. What about gender? Perhaps surprisingly, if we look at all males and all females, there isn’t a perceptible difference in the distributions of number of friends. But if we instead look at males and females as a function of age, there is a definite difference:
Teenage boys tend to have more friends than teenage girls, perhaps because they are less selective in who they accept as friends. But after the early 20s, the difference between genders rapidly dwindles.
What effect does relationship status have? Here’s the male and female data as a function of age:
In the older set, relationship status doesn’t seem to make much difference. But for young people it does. With teenagers who (mis)report themselves as “married” on average having more friends than those who don’t. And with early teenage girls who say they’re “engaged” (perhaps to be able to tag a BFF) typically having more friends than those who say they’re single, or just “in a relationship”.
Another thing that’s fairly reliably reported by Facebook users is location. And it’s common to see quite a lot of variation by location. Like here are comparisons of the median number of friends for countries around the world (ones without enough data are left gray), and for states in the US:
There are some curious effects. Countries like Russia and China have low median friend counts because Facebook isn’t widely used for connections between people inside those countries. And perhaps there are lower friend counts in the western US because of lower population densities. But quite why there are higher friend counts for our Facebook population in places like Iceland, Brazil and the Philippines—or Mississippi—I don’t know. (There is of course some “noise” from people misreporting their locations. But with the size of the sample we have, I don’t think this is a big effect.)
In Facebook, people can list both a “hometown” and a “current city”. Here’s how the probability that these are in the same US state varies with age:
What we see is pretty much what one would expect. For some fraction of the population, there’s a certain rate of random moving, visible here for young ages. Around age 18, there’s a jump as people move away from their “hometowns” to go to college and so on. Later, some fraction move back, and progressively consider wherever they live to be their “hometown”.
One can ask where people move to and from. Here’s a plot showing the number of people in our Facebook population moving between different US states, and different countries:
There’s a huge range of demographic questions we could ask. But let’s come back to social networks. It’s a common observation that people tend to be friends with people who are like them. So to test this we might for example ask whether people with more friends tend to have friends who have more friends. Here’s a plot of the median number of friends that our users have, as a function of the number of friends that they themselves have: 
And the result is that, yes, on average people with more friends tend to have friends with more friends. Though we also notice that people with lots of friends tend to have friends with fewer friends than themselves.
And seeing this gives me an opportunity to discuss a subtlety I alluded to earlier. The very first plot in this post shows the distribution of the number of friends that our users have. But what about the number of friends that their friends have? If we just average over all the friends of all our users, this is how what we get compares to the original distribution for our users themselves:
It seems like our users’ friends always tend to have more friends than our users themselves. But actually from the previous plot we know this isn’t true. So what’s going on? It’s a slightly subtle but general social-network phenomenon known as the “friendship paradox”. The issue is that when we sample the friends of our users, we’re inevitably sampling the space of all Facebook users in a very non-uniform way. In particular, if our users represent a uniform sample, any given friend will be sampled at a rate proportional to how many friends they have—with the result that people with more friends are sampled more often, so the average friend count goes up.
It’s perfectly possible to correct for this effect by weighting friends in inverse proportion to the number of friends they have—and that’s what we did earlier in this post. And by doing this we determine that in fact the friends of our users do not typically have more friends than our users themselves; instead their median number of friends is actually 229 instead of 342.
It’s worth mentioning that if we look at the distribution of number of friends that we deduce for the Facebook population, it’s a pretty good fit to a power law, with exponent -2.8. And this is a common form for networks of many kinds—which can be understood as the result of an effect known as “preferential attachment”, in which as the network grows, nodes that already have many connections preferentially get more connections, leading to a limiting “scale-free network” with power-law features.
But, OK. Let’s look in more detail at the social network of an individual user. I’m not sufficiently diligent on Facebook for my own network to be interesting. But my 15-year-old daughter Catherine was kind enough to let me show her network:
There’s a dot for each of Catherine’s Facebook friends, with connections between them showing who’s friends with whom. (There’s no dot for Catherine herself, because she’d just be connected to every other dot.) The network is laid out to show clusters or “communities” of friends (using the Wolfram Language function
FindGraphCommunities). And it’s amazing the extent to which the network “tells a story”. With each cluster corresponding to some piece of Catherine’s life or history.
Here’s a whole collection of networks from our Data Donors:
No doubt each of these networks tells a different story. But we can still generate overall statistics. Like, for example, here is a plot of how the number of clusters of friends varies with age (there’d be less noise if we had more data):
Even at age 13, people typically seem to have about 3 clusters (perhaps school, family and neighborhood). As they get older, go to different schools, take jobs, and so on, they accumulate another cluster or so. Right now the number saturates above about age 30, probably in large part just because of the limited time Facebook has been around.
How big are typical clusters? The largest one is usually around 100 friends; the plot below shows the variation of this size with age:
And here’s how the size of the largest cluster as a fraction of the whole network varies with age:
What about more detailed properties of networks? Is there a kind of “periodic table” of network structures? Or a classification scheme like the one I made long ago for cellular automata?
The first step is to find some kind of iconic summary of each network, which we can do for example by looking at the overall connectivity of clusters, ignoring their substructure. And so, for example, for Catherine (who happened to suggest this idea), this reduces her network to the following “cluster diagram”:
Doing the same thing for the Data Donor networks shown above, here’s what we get:
In making these diagrams, we’re keeping every cluster with at least 2 friends. But to get a better overall view, we can just drop any cluster with, say, less than 10% of all friends—in which case for example Catherine’s cluster diagram becomes just:
And now for example we can count the relative numbers of different types of structures that appear in all the Data Donor networks:
And we can look at how the fractions of each of these structures vary with age:
What do we learn? The most common structures consist of either two or three major clusters, all of them connected. But there are also structures in which major clusters are completely disconnected—presumably reflecting facets of a person’s life that for reasons of geography or content are also completely disconnected.
For everyone there’ll be a different detailed story behind the structure of their cluster diagram. And one might think this would mean that there could never be a general theory of such things. At some level it’s a bit like trying to find a general theory of human history, or a general theory of the progression of biological evolution. But what’s interesting now about the Facebook world is that it gives us so much more data from which to form theories.
And we don’t just have to look at things like cluster diagrams, or even friend networks: we can dig almost arbitrarily deep. For example, we can analyze the aggregated text of posts people make on their Facebook walls, say classifying them by topics they talk about (this uses a natural-language classifier written in the Wolfram Language and trained using some large corpora):
Each of these topics is characterized by certain words that appear with high frequency:
And for each topic we can analyze how its popularity varies with (Facebook) age:
It’s almost shocking how much this tells us about the evolution of people’s typical interests. People talk less about video games as they get older, and more about politics and the weather. Men typically talk more about sports and technology than women—and, somewhat surprisingly to me, they also talk more about movies, television and music. Women talk more about pets+animals, family+friends, relationships—and, at least after they reach child-bearing years, health. The peak time for anyone to talk about school+university is (not surprisingly) around age 20. People get less interested in talking about “special occasions” (mostly birthdays) through their teens, but gradually gain interest later. And people get progressively more interested in talking about career+money in their 20s. And so on. And so on.
Some of this is rather depressingly stereotypical. And most of it isn’t terribly surprising to anyone who’s known a reasonable diversity of people of different ages. But what to me is remarkable is how we can see everything laid out in such quantitative detail in the pictures above—kind of a signature of people’s thinking as they go through life.
Of course, the pictures above are all based on aggregate data, carefully anonymized. But if we start looking at individuals, we’ll see all sorts of other interesting things. And for example personally I’m very curious to analyze my own archive of nearly 25 years of email—and then perhaps predict things about myself by comparing to what happens in the general population.
Over the decades I’ve been steadily accumulating countless anecdotal “case studies” about the trajectories of people’s lives—from which I’ve certainly noticed lots of general patterns. But what’s amazed me about what we’ve done over the past few weeks is how much systematic information it’s been possible to get all at once. Quite what it all means, and what kind of general theories we can construct from it, I don’t yet know.
But it feels like we’re starting to be able to train a serious “computational telescope” on the “social universe”. And it’s letting us discover all sorts of phenomena. That have the potential to help us understand much more about society and about ourselves. And that, by the way, provide great examples of what can be achieved with data science, and with the technology I’ve been working on developing for so long.
viernes, 26 de abril de 2013
La dinámica de redes de los alias
LulzOpSec: The Network Dynamics of Aliases
Whoa! "Anonymous," a/k/a "the coolest distributed, decentralized, leaderless, amorphous transnational activist network since al Qaeda," recently suffered a major blow when its alleged leader, Hector Xavier Monsegur, sold out the group's elite talent to the FBI in order to save his own hide. This is pretty exciting news and will be a fun, recurring SNA case study.
The big question folks will be asking is "What does this mean for Anonymous?" We should already know the answer to this one, because we have gotten so good at answering the question "What does this mean for al Qaeda?" This is just another decapitation: a "hierarchy" strategy in a "network" age. Illicit networks with tangible, hierarchical command and control infrastructures are capable of sophisticated, synchronized, and totally sweet attacks (or hacks), but are prone to detection and infiltration. Loosely organized, decentralized, leaderless illicit networks enjoy greater operational security and penetration, but their attacks (or hacks), in comparison, are pretty lame. "Decapitation" basically transforms the former into the latter.
Anonymous just lost its command and control infrastructure, as well as considerable expertise, as a result of the FBI's hard work. Anonymous is now a flatter organization; a decentralized and distributed network of mostly posers, wannabes, and copycats. This group of second stringers is probably nowhere near as skilled or sophisticated as the core group just arrested. But cybersecurity professionals must not get cocky. Months after 9/11, Afghanistan was invaded and al Qaeda was blasted back to the Stone Age. Instead of going extinct, al Qaeda evolved: it began as an organization and transformed into a movement. What will Anonymous look or sound like from this point forward? This post, in the comments section at Gizmodo, eloquently speaks to this network's next evolution:
************
Now that I got that out of the way, I am ready to do some serious SNA work. In this post, I hope to accomplish a few things:
1) Illustrate how aliases facilitate an illicit cell's operational security
2) Illustrate how network analytical techniques can be used to defeat this maneuver
3) Suggest how an illicit cell might evolve to be resilient against these analytical techniques
1. Aliases facilitate operational security
The indictment of the LulzSec/Anonymous crew reads a little something like this [liberties taken]:
"RYAN ACKROYD, a/k/a “kayla,” a/k/a “lol,” a/k/a “lolspoon”; JAKE DAVIS, a/k/a “topiary,” a/k/a “atopiary”; DARREN MARTYN, a/k/a “pwnsauce,” a/k/a “raepsauce,” a/k/a “networkkitten”; and DONNCHA O’CEARRBHAIL, a/k/a “palladium," [were ratted out by] "HECTOR XAVIER MONSEGUR, a/k/a “Sabu,” a/k/a “Xavier DeLeon,” a/k/a “Leon[.]”
What's with all these aliases? Well, let's first take a look at this network without the benefit of aliases:
Not a whole lot to see here, is there? The sociogram and accompanying sociomatrix above indicate a pretty straightforward network. There is no security inherent in the network structure. Instead, Anonymous relied on operational security measures both high tech (e.g. proxy servers) and low tech (e.g. aliases) in order to assure its members' continued... ahem... anonymity.
Wait a minute: Aliases? Seriously? In a world of cryptography, plastic surgery, stealth drones, SIGINT, facial recognition software and Stuxnet, I can live off the grid if I just call myself Toby instead of Chad? Well, yeah, sorta'. If I say, "Hello, this is CHAD" when I am on the phone, and Twitter from @CHAD, and my email is CHAD@email, then virtually all of my communications, regardless of their medium, will be directed ties originating from a single network node called CHAD. I have made it really easy for the federali to creep on me. However, if my phone is registered to Toby, I Twitter from @Jimbo, and my email is Bertha@email, I have created the impression of 3 independent nodes, each with their own network dimensions. Let's take a look at what happens to the tiny LulzSec network when we insert these guys' aliases into the network:
2. Aliases and their owners are structurally equivalent
We all know and love OrgCharts. They remind us of how inconsequential and utterly replaceable we all are. In SNA, we like to refer to that replaceability as structural equivalency. Look at the nuanced and sensitive orgchart above. Structurally, what does a Leader look like? Well, he is linked to Lieutenants, but not Losers. He cannot stand the smell of the proletariat. What does a Loser look like? Well, he is linked to Lieutenants and fellow Losers, but not Leaders. He's never met a leader he didn't want to stab with a pitchfork. What does a Lieutenant look like? Well, he is linked to Leaders, fellow Lieutenants, and Losers. He pretty much has the worst job in the world and everyone hates him. Anyways, this should all be pretty elementary.
Now let's say we insert a random person into this OrgChart. His name is Toby. We do not know if he is a Leader, Lieutenant, or Loser. So then we ask, "What do his links look like?" If the answer is "He is linked to Leaders, Lieutenants, and Losers," then we know he is a Lieutenant. If the answer is "He is only linked to Lieutenants," then we know he is a Leader. If the answer is "He is only linked to Lieutenants and Losers," then we know he is a Loser. And that is structural equivalency in its most basic form.
To spot an alias (that is, a fake actor), we have to take structural equivalency one step further.
Let's say we determine that "Toby" is a Lieutenant. We notice that "Toby," like his fellow Liutenant "Chad," is linked to Leader "Bertha" and Losers "Larry," "Curley," and "Moe." But then we notice something weird: "Chad" and "Toby" are all linked to the same exact people, but they are NOT LINKED TO EACH OTHER. This network reality could indicate that "Toby" is an alias for "Chad," or vice versa. Why? Because no one, unless mentally ill, has conversations with their alias. If you are using an alias for cover, you are doing everything in your power to separate your real identity from your fake one. That distinctive gap, or lack of a link, can be very telling.
Let's break this on down, SNA-style:
In the sociomatrix above, we are looking at the LulzSec network, including aliases. The "1.000" indicates a link between actors when the " " indicates that there is no link between actors. Let's drill down on the actors "HECTOR_MONSEGUR," "leon," "xavier_deleon," and "sabu." They enjoy links with everyone in this network, but seem to want nothing to do with each other. Why? Because they are the same dude. The other actors and their aliases are spotted the same way.
The sociomatrix above presents a crystal clear picture: what began as 17 actors has been condensed down to 5. Actual actors have been grouped together with their structurally equivalent aliases. The jig is up. For this we can thank an SNA technique called "Block-modeling," an algorithm that rearranges network actors and groups them into structurally equivalent "blocks." I will explore block-modeling further another time, but for now, know this:
3. Towards an SNA-resistant alias
There have been security professionals and dedicated public servants who have talked candidly about terrorist capabilities and have spelled out, explicitly, the vulnerabilities said terrorists could exploit. These people are sometimes accused of "providing a blueprint for terrorism." The folks who make these accusations think that the bad guys are knuckle-dragging, cave-dwelling medievalists. They are dead wrong. The real bad guys, the real threats, are the engineers, scientists, chemists, and managers that empower and compel the knuckle-dragging, cave-dwelling medievalists. These bad guys are way smarter than we are. When we talk about specific tactics that bad guys can use, we are merely talking about tactics that they have already spent years using, perfecting, considering, or passing over for an even better tactic.
That's out of the way, so here it is: SNA-resilient alias will talk to one another. My aliases "Toby" and "Steve" will chat with each other in public online spaces (Toby uses my desktop while Steve uses a mobile device). Both Toby and Steve will attend the same security conference: Toby will write an online review of the event while Steve links to it on his Facebook account. In fact, I will meet them both there and get lunch with them. The gaps and the contours of the network expose its structure and its inner workings. The gap will not hide you, it will only make you more distinct, isolated, and conspicuous. But the link can hide you. Each time aliases come together in manufactured ways, the link between them grows stronger. But enough appearances together, enough conversations amongst each other, enough co-appearances on the same roster... soon enough, they are mutually exclusive and distinct actors.... and somebody's case has gone to shit.
Third Degree Centrality
Suscribirse a:
Entradas (Atom)