Extraire des utilisateurs Twitter avec Curl

Bonjour à tous,

J’ai remarqué qu’il était possible d’extraire des informations sur les utilisateurs de Twitter sans Token ou vérifications de la part du serveur de Twitter. (Twitter considère que ces informations sont publiques et qu’il n’a pas à protéger l’extraction arbitraire des données) . Ce qui est intéressant dans le cas présent, c’est l’organisation de ces données.

Imaginons une requête classique avec curl :

curl -s 'https://syndication.twitter.com/widgets/followbutton/info.json?user_ids=2[0000-9999]'

Donnant le résultat suivant :

[][][][{"following":false,"id":"20003","screen_name":"steve_wing","name":"Steve Wing","protected":false,"followers_count":1575,"formatted_followers_count":"1,575 followers","age_gated":false}][][][][][][][][][][{"following":false,"id":"20013","screen_name":"downcorridors","name":"Jolene Thompson","protected":false,"followers_count":470,"formatted_followers_count":"470 followers","age_gated":false}][][][][][][][][][][{"following":false,"id":"20023","screen_name":"d4rr3ll","name":"Darrell Taylor","protected":false,"followers_count":373,"formatted_followers_count":"373 followers","age_gated":false}][][][][]

Maintenant, imaginons que l’on puisse classer et organiser le résultat :

La requête suivante ajoute un saut de ligne et ne récupère que les arrays valables supprimant ainsi les arrays vides.

curl 'https://syndication.twitter.com/widgets/followbutton/info.json?user_ids=2[0000-9999]' -s -w "\n" | grep "following"

Résultat :

[{"following":false,"id":"20003","screen_name":"steve_wing","name":"Steve Wing","protected":false,"followers_count":1575,"formatted_followers_count":"1,575 followers","age_gated":false}]
[{"following":false,"id":"20013","screen_name":"downcorridors","name":"Jolene Thompson","protected":false,"followers_count":470,"formatted_followers_count":"470 followers","age_gated":false}]
[{"following":false,"id":"20023","screen_name":"d4rr3ll","name":"Darrell Taylor","protected":false,"followers_count":373,"formatted_followers_count":"373 followers","age_gated":false}]
[{"following":false,"id":"20033","screen_name":"barryh","name":"Barry Hedges","protected":true,"followers_count":4,"formatted_followers_count":"4 followers","age_gated":false}]
[{"following":false,"id":"20043","screen_name":"wenxin","name":"Kevin \u6587\u5FC3","protected":false,"followers_count":4133,"formatted_followers_count":"4,133 followers","age_gated":false}]
[{"following":false,"id":"20053","screen_name":"onelonemonkey","name":"Alan White","protected":false,"followers_count":188,"formatted_followers_count":"188 followers","age_gated":false}]
[{"following":false,"id":"20063","screen_name":"shuckle","name":"Matt Machell","protected":false,"followers_count":834,"formatted_followers_count":"834 followers","age_gated":false}]
[{"following":false,"id":"20073","screen_name":"Remy_Zane","name":"Remy Zane","protected":false,"followers_count":527,"formatted_followers_count":"527 followers","age_gated":false}]
[{"following":false,"id":"20083","screen_name":"BrightonRecords","name":"Jeff Thompson","protected":false,"followers_count":29,"formatted_followers_count":"29 followers","age_gated":false}]
[{"following":false,"id":"20093","screen_name":"negster","name":"n3g","protected":false,"followers_count":43,"formatted_followers_count":"43 followers","age_gated":false}]
[{"following":false,"id":"20103","screen_name":"pons","name":"Sandro Golinelli","protected":false,"followers_count":56,"formatted_followers_count":"56 followers","age_gated":false}]
[{"following":false,"id":"20113","screen_name":"PixelNurse","name":"Lee J Parry","protected":false,"followers_count":342,"formatted_followers_count":"342 followers","age_gated":false}]
[{"following":false,"id":"20123","screen_name":"christina_sea","name":"christina_sea","protected":false,"followers_count":3,"formatted_followers_count":"3 followers","age_gated":false}]
[{"following":false,"id":"20133","screen_name":"happeningfish","name":"Jamie \uD83E\uDD8B MacDonald","protected":false,"followers_count":893,"formatted_followers_count":"893 followers","age_gated":false}]
[{"following":false,"id":"20143","screen_name":"nemokid","name":"fabio","protected":false,"followers_count":290,"formatted_followers_count":"290 followers","age_gated":false}]
[{"following":false,"id":"20153","screen_name":"karihelened","name":"karoline","protected":false,"followers_count":33,"formatted_followers_count":"33 followers","age_gated":false}]
[{"following":false,"id":"20163","screen_name":"marksheldon","name":"Mark Sheldon","protected":false,"followers_count":653,"formatted_followers_count":"653 followers","age_gated":false}]
[{"following":false,"id":"20173","screen_name":"sawtopher","name":"Christopher Saw","protected":true,"followers_count":48,"formatted_followers_count":"48 followers","age_gated":false}]
[{"following":false,"id":"20183","screen_name":"MiMo","name":"MiMo","protected":false,"followers_count":327,"formatted_followers_count":"327 followers","age_gated":false}]
[{"following":false,"id":"20193","screen_name":"Kiltboyo","name":"Kiltboyo","protected":false,"followers_count":26,"formatted_followers_count":"26 followers","age_gated":false}]
[{"following":false,"id":"20203","screen_name":"blethers","name":"Christine McIntosh","protected":false,"followers_count":546,"formatted_followers_count":"546 followers","age_gated":false}]
[{"following":false,"id":"20213","screen_name":"imlxgr","name":"Lukas","protected":false,"followers_count":69,"formatted_followers_count":"69 followers","age_gated":false}]

Utilisant JQ pour formater les informations.

Pour installer JQ sur Mac en passant par Brew, c’est : brew install jq

curl 'https://syndication.twitter.com/widgets/followbutton/info.json?user_ids=2[0000-0009]' -s -w "\n" | jq

Le résultat est le suivant :

[]
[]
[]
[
  {
    "following": false,
    "id": "20003",
    "screen_name": "steve_wing",
    "name": "Steve Wing",
    "protected": false,
    "followers_count": 1575,
    "formatted_followers_count": "1,575 followers",
    "age_gated": false
  }
]
[]
[]
[]
[]
[]
[]

Optimisons maintenant la requête JQ pour ne garder que l’essentiel :

curl 'https://syndication.twitter.com/widgets/followbutton/info.json?user_ids=2[0000-0009]' -s -w "\n" | jq '.[]'
{
  "following": false,
  "id": "20003",
  "screen_name": "steve_wing",
  "name": "Steve Wing",
  "protected": false,
  "followers_count": 1575,
  "formatted_followers_count": "1,575 followers",
  "age_gated": false
}

Extraction du nom d’utilisateur.

curl 'https://syndication.twitter.com/widgets/followbutton/info.json?user_ids=2[0000-0009]' -s -w "\n" | jq '.[].screen_name'

Le résultat devient :

"steve_wing"

Note de fin :

Il ne vous reste plus qu’à tester différents paramètres JQ avec différentes valeurs d’user_ids.

Pour vous faire une idée de ce que cela donne, je vous propose une vidéo sur YouTube que je viens de réaliser :

Pour comprendre et améliorer votre technique en JQ, le site https://jqplay.org est une mine d’information.

Rbcafe

Leave a Comment