Exception ignored. RuntimeError: Event loop is closed - problem with the async await branch #14

ihabpalamino · 2023-07-26T17:08:31Z

while trying to scrape using the branch async_await i got the result that i have wanted but i got this error

API Updated Successfully.
Successfully Logged In..
["Opération anti-terroriste: Les perquisitions effectuées aux domiciles de certains mis en cause ont permis la saisie d'armes blanches, d'ouvrages et documents à contenu extrémiste et de guides de fabrication d'engins explosifs.\n#Maroc https://t.co/fxoAGZ5KuB", "🔴 Coup de filet anti-terroriste #BCIJ / #DGST/ #BNPJ: Une cinquantaine de personnes ont été interpellées, aujourd'hui, une vingtaine placées en garde à vue, suite à des opérations sécuritaires simultanées menées dans plusieurs villes du #Maroc.\n@DGSN_MAROC https://t.co/ZI40SYkDez", 'بعد توقيف المشتبه فيهم نفذت المصالح الأمنية وعناصر مراقبة التراب الوطني عمليات التفتيش بمنازل الموقوفين، والتي أدى بعضها إلى اكتشاف أسلحة بيضاء عبارة عن سيوف وفؤوس وهراوات ذات استخدام مشبوه. بالإضافة إلى كتب ومخطوطات تحتوي على خطب مع تحرض على "الجهاد"، وتتضمن أيضا وثائق تبين… https://t.co/3OewK8ULmW', '🔴 نفذت مصالح الامن  يومه الاربعاء عملية أمنية في صفوف عدد من المشتبهين في أوساط التيار المتطرف.\n\nوأطاحت هذه العملية الأمنية بحوالي 50 شخصا تمت إحالتهم على التحقيق تحت إشراف النيابة العامة المختصة.\n\n#المغرب https://t.co/EO8EMrywJM', "#Fête_du_Trône #Maroc\n\nÉnergies renouvelables: En quelques années, le #Maroc s'est positionné en leader mondial à la faveur d'une politique volontariste portée au plus haut sommet de l'Etat, des stratégies ciblées et des mégaprojets structurants. https://t.co/YeT8gI8cfE", "L'ancien gouverneur de Bank Al-Maghrib, Mohamed Sekkat est décédé, mercredi à Rabat, à l'âge de 81 ans.\n#Maroc https://t.co/7KaQoxDVer", '#المغرب #عيد_العرش \n\nتعزيز و تطوير البنيات الصحية في مختلف مناطق المملكة يعكس حرص صاحب الجلالة الملك محمد السادس على توفير خدمات طبية لمختلف فئات المجتمع https://t.co/cNY40Dqg5s', '#المغرب #عيد_العرش \n\nتعميم الحماية الاجتماعية ورش مجتمعي غير مسبوق. هذا المشروع الذي ترأس حفل إطلاق تنزيله صاحب الجلالة الملك محمد السادس في 14 أبريل 2020، يأتي ليثمن الإنجازات الهامة التي راكمها المغرب في هذا المجال https://t.co/MJ0KYJVxE3', '#عيد_العرش #المغرب\n\n استثمارات ضخمة، مشاريع مهيكلة ورؤية طموحة...الطاقات المتجددة، أساس استراتيجية المغرب الطاقية. ورش يحظى بالعناية السامية لصاحب الجلالة الملك محمد السادس. https://t.co/l6EMQTLZPl', '📌 📺 الطاقات المتجددة، تحدي القرن الحادي والعشرين.. ورش ملكي كبير أسس ل«\xa0ثورة طاقية» بالمملكة. \n\nبمناسبة عيد العرش، القناة الثانية تحكي لكم هذه الثورة، التي منحت المغرب الريادة في مجال الطاقات النظيفة، وذلك عبر وثائقي معزز بشهادات نادرة وصور أخاذة.\n\nموعدكم  مع «\xa0 المغرب، المملكة… https://t.co/PgS9Um1lap', "📺 Les énergies renouvelables, un enjeu du 21ème siècle. Au #Maroc, c'est un chantier de Règne. La Révolution énergétique du Royaume est en marche.\n\nA l'occasion de la #Fête_du_Trône, 2M vous raconte ce Maroc, leader des énergies propres, à travers un récit inédit fait de… https://t.co/80wwiU81YF", "Le nombre de clients du Groupe #Maroc Telecom a atteint près de 75 millions à fin juin 2023, en légère baisse de 0,5% par rapport à la même période de l'année précédente. https://t.co/7khuFJ55Lv"]
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x00000267CB2F3820>
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 751, in call_soon
    self._check_closed()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

here is my code

from tweeterpy import TweeterPy
from tweeterpy import util

twitter = TweeterPy()

twitter.login("***","****")
# OR generate session manually with auth-token (if if you're unable to log in)
# twitter.generate_session(auth_token="auth_token_here")
userid=twitter.get_user_id("2MInteractive")
# set with_replies to True if you are also interested in tweet replies, if you only need tweets, leave it as it is. Feel free to see total number of tweets you want to get, but default it gets all if set to None.
user_tweets = twitter.get_user_media(userid,from_date="2023-07-25",to_date="2023-07-26")
usernames = util.find_nested_key(user_tweets,"full_text")


print(usernames)

iSarabjitDhiman · 2023-07-27T05:35:28Z

Nothing to worry about. Just check if you got the result you wanted. Try running it again, seems like a connection lost error to me. If the error still persists let me know I will look into it.
Also you have copied your id password along with the error log up there, please change your id password and edit the message to remove it.

ihabpalamino · 2023-07-27T08:40:49Z

the error is still pesisting ;

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x00000131DBEB3820>
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 751, in call_soon
    self._check_closed()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

iSarabjitDhiman · 2023-07-27T17:54:07Z

Hey @ihabpalamino ,
I just tested it on my side, it seems to be working fine for me. I just noticed that you may have embedded/imported this code to some of your project, is there any chance that the error belongs to some of your code? Because from the first message you sent, I can tell that the code returned the data and also filtered for the "full_text" key. Try running the Tweeterpy code in a separate terminal and see if the error still persists. Let me know how it goes. Feel free to email me a screen recording, it might help. Or you can also contact me on social media.

ihabpalamino · 2023-07-28T10:11:42Z

Hey @ihabpalamino , I just tested it on my side, it seems to be working fine for me. I just noticed that you may have embedded/imported this code to some of your project, is there any chance that the error belongs to some of your code? Because from the first message you sent, I can tell that the code returned the data and also filtered for the "full_text" key. Try running the Tweeterpy code in a separate terminal and see if the error still persists. Let me know how it goes. Feel free to email me a screen recording, it might help. Or you can also contact me on social media.

here is my code

import pandas as pd
import json  # Import the json module

from tweeterpy import TweeterPy, util

twitter = TweeterPy()
twitter.login("*****", "**** ")

data = []
userid = twitter.get_user_id("2MInteractive")

user_tweets = twitter.get_user_tweets(userid, from_date="2023-07-24", to_date="2023-07-27",total=5)
for result in user_tweets:
    for tweet_entry in result['data']:
        tweet_data = tweet_entry.get('content', {}).get('itemContent', {}).get('tweet_results', {}).get('result', {}).get('legacy', {})
        created_at_list = tweet_data.get('created_at', [])

        if len(created_at_list) >= 2:
            second_created_at = created_at_list
        else:
            second_created_at = None

        # Append the data to the existing list for the current tweet
        data.append({
            "id_post":util.find_nested_key(tweet_entry, "screen_name"),
            "username": util.find_nested_key(tweet_entry, "screen_name"),
            "content":  util.find_nested_key(tweet_entry, "full_text"),
            "Date": second_created_at,
            "viewVideo": util.find_nested_key(tweet_entry, "viewCount"),
            'likecount': util.find_nested_key(tweet_entry, "favorite_count"),
            'shares': util.find_nested_key(tweet_entry, "retweet_count"),
            'comments': util.find_nested_key(tweet_entry, "reply_count"),
            'postUrl':util.find_nested_key(tweet_entry, "expanded_url"),
            "platformname":"twitter"
        })

# Create a pandas DataFrame from the data list
df = pd.DataFrame(data, columns=["id_post", "username","content", "Date" ,"viewVideo",'likecount', 'shares','comments','postUrl', "platformname"])

# Convert DataFrame to JSON format and display it
data_json = df.to_json(orient='records', indent=4, force_ascii=False)
print(data_json)

# Save the DataFrame to a CSV file
df.to_csv('C:/Users/HP Probook/OneDrive/Images/Bureau/CsvScraping/twitter.csv', index=False, encoding='utf-8')

iSarabjitDhiman · 2023-07-28T10:38:07Z

Hey @ihabpalamino , I just tested it on my side, it seems to be working fine for me. I just noticed that you may have embedded/imported this code to some of your project, is there any chance that the error belongs to some of your code? Because from the first message you sent, I can tell that the code returned the data and also filtered for the "full_text" key. Try running the Tweeterpy code in a separate terminal and see if the error still persists. Let me know how it goes. Feel free to email me a screen recording, it might help. Or you can also contact me on social media.

here is my code

import pandas as pd
import json  # Import the json module

from tweeterpy import TweeterPy, util

twitter = TweeterPy()
twitter.login("*****", "**** ")

data = []
userid = twitter.get_user_id("2MInteractive")

user_tweets = twitter.get_user_tweets(userid, from_date="2023-07-24", to_date="2023-07-27",total=5)
for result in user_tweets:
    for tweet_entry in result['data']:
        tweet_data = tweet_entry.get('content', {}).get('itemContent', {}).get('tweet_results', {}).get('result', {}).get('legacy', {})
        created_at_list = tweet_data.get('created_at', [])

        if len(created_at_list) >= 2:
            second_created_at = created_at_list
        else:
            second_created_at = None

        # Append the data to the existing list for the current tweet
        data.append({
            "id_post":util.find_nested_key(tweet_entry, "screen_name"),
            "username": util.find_nested_key(tweet_entry, "screen_name"),
            "content":  util.find_nested_key(tweet_entry, "full_text"),
            "Date": second_created_at,
            "viewVideo": util.find_nested_key(tweet_entry, "viewCount"),
            'likecount': util.find_nested_key(tweet_entry, "favorite_count"),
            'shares': util.find_nested_key(tweet_entry, "retweet_count"),
            'comments': util.find_nested_key(tweet_entry, "reply_count"),
            'postUrl':util.find_nested_key(tweet_entry, "expanded_url"),
            "platformname":"twitter"
        })

# Create a pandas DataFrame from the data list
df = pd.DataFrame(data, columns=["id_post", "username","content", "Date" ,"viewVideo",'likecount', 'shares','comments','postUrl', "platformname"])

# Convert DataFrame to JSON format and display it
data_json = df.to_json(orient='records', indent=4, force_ascii=False)
print(data_json)

# Save the DataFrame to a CSV file
df.to_csv('C:/Users/HP Probook/OneDrive/Images/Bureau/CsvScraping/twitter.csv', index=False, encoding='utf-8')

Let me make some changes to the code, I will let u know when it's done.

Edit : Seems like there is some issue with Windows. I was testing it on ubuntu so that might be the reason I didn't get the error.

For the time being, here is a quick fix if you want to apply, otherwise I will update the code shortly.

ihabpalamino · 2023-07-28T11:34:52Z

thanks waiting for you to resolve the issue

iSarabjitDhiman · 2023-07-29T14:20:59Z

Hi @ihabpalamino
Could you please confirm if the issue has been resolved? I just made changes to the code. Test it and let me know how it goes.

ihabpalamino · 2023-07-30T11:16:03Z

Hi @ihabpalamino Could you please confirm if the issue has been resolved? I just made changes to the code. Test it and let me know how it goes.

okey i will check if is it possible to push the date since until to the master branch because it works fine already

ihabpalamino · 2023-07-31T08:18:40Z

i have tried it and it does works fine no more error thank you sir so much

iSarabjitDhiman · 2023-07-31T08:49:52Z

i have tried it and it does works fine no more error thank you sir so much

As the issue seems to have been resolved after this commit 96275ad , I am closing the issue.
Also will merge the async-await branch into the master soon, just working on some minor bugs before it's merged.

ihabpalamino · 2023-07-31T09:21:55Z

thank u so much hope it wont take a lot of time really need it for my final project studies thank u for all ur hard work

iSarabjitDhiman changed the title ~~problem with the async await branch~~ Exception ignored. RuntimeError: Event loop is closed - problem with the async await branch Jul 28, 2023

iSarabjitDhiman closed this as completed Jul 31, 2023

iSarabjitDhiman added bug Something isn't working enhancement New feature or request and removed bug Something isn't working enhancement New feature or request labels Aug 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exception ignored. RuntimeError: Event loop is closed - problem with the async await branch #14

Exception ignored. RuntimeError: Event loop is closed - problem with the async await branch #14

ihabpalamino commented Jul 26, 2023 •

edited by iSarabjitDhiman

iSarabjitDhiman commented Jul 27, 2023

ihabpalamino commented Jul 27, 2023 •

edited by iSarabjitDhiman

iSarabjitDhiman commented Jul 27, 2023 •

edited

ihabpalamino commented Jul 28, 2023 •

edited by iSarabjitDhiman

iSarabjitDhiman commented Jul 28, 2023 •

edited

ihabpalamino commented Jul 28, 2023

iSarabjitDhiman commented Jul 29, 2023

ihabpalamino commented Jul 30, 2023

ihabpalamino commented Jul 31, 2023

iSarabjitDhiman commented Jul 31, 2023

ihabpalamino commented Jul 31, 2023

Exception ignored. RuntimeError: Event loop is closed - problem with the async await branch #14

Exception ignored. RuntimeError: Event loop is closed - problem with the async await branch #14

Comments

ihabpalamino commented Jul 26, 2023 • edited by iSarabjitDhiman

iSarabjitDhiman commented Jul 27, 2023

ihabpalamino commented Jul 27, 2023 • edited by iSarabjitDhiman

iSarabjitDhiman commented Jul 27, 2023 • edited

ihabpalamino commented Jul 28, 2023 • edited by iSarabjitDhiman

iSarabjitDhiman commented Jul 28, 2023 • edited

ihabpalamino commented Jul 28, 2023

iSarabjitDhiman commented Jul 29, 2023

ihabpalamino commented Jul 30, 2023

ihabpalamino commented Jul 31, 2023

iSarabjitDhiman commented Jul 31, 2023

ihabpalamino commented Jul 31, 2023

ihabpalamino commented Jul 26, 2023 •

edited by iSarabjitDhiman

ihabpalamino commented Jul 27, 2023 •

edited by iSarabjitDhiman

iSarabjitDhiman commented Jul 27, 2023 •

edited

ihabpalamino commented Jul 28, 2023 •

edited by iSarabjitDhiman

iSarabjitDhiman commented Jul 28, 2023 •

edited