Python: Naskah A Google Autosuggest Extract of Trends for your Niche Search Keywords

Script Python Pikeun Nangkep Tren Autosuggest

Sarerea resep Google Trends, tapi éta rada hésé lamun ngeunaan Long Tail Keywords. Urang sadayana resep pejabat jasa tren google pikeun kéngingkeun wawasan ngeunaan kabiasaan milarian. Nanging, dua hal nyegah seueur ngagunakeunana pikeun padet;

  1. Nalika anjeun kedah milarian kecap konci Ecological anyar, Tuh henteu cekap data dina Google Trends 
  2. Kurangna API resmi pikeun nyungkeun permintaan kana tren google: Nalika urang nganggo modul sapertos pytrends, maka urang kedah nganggo server proxy, atanapi urang dipeungpeuk. 

Dina tulisan ieu, kuring bakal ngabagi Skrip Python kami parantos nyerat pikeun ngekspor kecap konci trending ngalangkungan Google Autosuggest.

Candak sareng Simpen Hasil Autosuggest Langkung Liwat 

Anggap urang gaduh 1,000 kecap konci Cikal pikeun dikirim ka Google Autosuggest. Salaku balesan, urang sigana bakal kéngingkeun 200,000 longtail kecap konci. Teras, urang kedah ngalakukeun hal anu sami saminggu engké sareng ngabandingkeun set data ieu pikeun ngajawab dua patarosan:

  • Pertanyaan anu mana kecap konci anyar dibandingkeun sareng terakhir waktos? Ieu panginten anu diperyogikeun ku urang. Google mikirkeun pamundut éta janten langkung penting - ku ngalakukeun éta, urang tiasa nyiptakeun solusi Google Autosuggest urang sorangan! 
  • Pertanyaan anu mana kecap konci henteu deui tren?

Naskahna rada gampang, sareng kaseueuran kode anu kuring bagi Ieuh. Kodeu anu diénggalan nyimpen data tina jalan anu lalu sareng ngabandingkeun saran kana waktos. Kami nyingkahan basis data basis file sapertos SQLite pikeun ngajantenkeun saderhana - janten sadayana panyimpenan data nganggo file CSV di handap. Ieu ngamungkinkeun anjeun ngimpor file dina Excel sareng ngajajah tren kata kunci khusus pikeun bisnis anjeun.

Pikeun Ngamangpaatkeun Script Python Ieu

  1. Lebetkeun sét kata kunci siki anjeun anu kedah dikirimkeun kana autocomplete: keywords.csv
  2. Saluyukeun setelan Script pikeun kabutuhan anjeun:
    • BAHASA: standar “en”
    • NEGARA: standar "kami"
  3. Jadwalkeun skrip na dijalankeun saminggu sakali. Anjeun tiasa ogé ngajalankeunana sacara manual sakumaha anu anjeun pikahoyong.
  4. Anggo keyword_suggestions.csv pikeun analisa salajengna:
    • kahiji_seen: ieu mangrupikeun tanggal dimana pamundut nembongan pikeun anu munggaran dina otomatis
    • last_seen: tanggal dimana pamundut ditingali pikeun terakhir kalina
    • nyaéta_baru: upami first_seen == last_seen kami atur ieu leres - Ngan tapiskeun nilai ieu pikeun kéngingkeun milarian tren anu anyar dina Google autosuggest.

Ieu Konci Python

# Pemavor.com Autocomplete Trends
# Author: Stefan Neefischer (stefan.neefischer@gmail.com)
import concurrent.futures
from datetime import date
from datetime import datetime
import pandas as pd
import itertools
import requests
import string
import json
import time

charList = " " + string.ascii_lowercase + string.digits

def makeGoogleRequest(query):
    # If you make requests too quickly, you may be blocked by google 
    time.sleep(WAIT_TIME)
    URL="http://suggestqueries.google.com/complete/search"
    PARAMS = {"client":"opera",
            "hl":LANGUAGE,
            "q":query,
            "gl":COUNTRY}
    response = requests.get(URL, params=PARAMS)
    if response.status_code == 200:
        try:
            suggestedSearches = json.loads(response.content.decode('utf-8'))[1]
        except:
            suggestedSearches = json.loads(response.content.decode('latin-1'))[1]
        return suggestedSearches
    else:
        return "ERR"

def getGoogleSuggests(keyword):
    # err_count1 = 0
    queryList = [keyword + " " + char for char in charList]
    suggestions = []
    for query in queryList:
        suggestion = makeGoogleRequest(query)
        if suggestion != 'ERR':
            suggestions.append(suggestion)

    # Remove empty suggestions
    suggestions = set(itertools.chain(*suggestions))
    if "" in suggestions:
        suggestions.remove("")
    return suggestions

def autocomplete(csv_fileName):
    dateTimeObj = datetime.now().date()
    #read your csv file that contain keywords that you want to send to google autocomplete
    df = pd.read_csv(csv_fileName)
    keywords = df.iloc[:,0].tolist()
    resultList = []

    with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
        futuresGoogle = {executor.submit(getGoogleSuggests, keyword): keyword for keyword in keywords}

        for future in concurrent.futures.as_completed(futuresGoogle):
            key = futuresGoogle[future]
            for suggestion in future.result():
                resultList.append([key, suggestion])

    # Convert the results to a dataframe
    suggestion_new = pd.DataFrame(resultList, columns=['Keyword','Suggestion'])
    del resultList

    #if we have old results read them
    try:
        suggestion_df=pd.read_csv("keyword_suggestions.csv")
        
    except:
        suggestion_df=pd.DataFrame(columns=['first_seen','last_seen','Keyword','Suggestion'])
    
    suggestionCommon_list=[]
    suggestionNew_list=[]
    for keyword in suggestion_new["Keyword"].unique():
        new_df=suggestion_new[suggestion_new["Keyword"]==keyword]
        old_df=suggestion_df[suggestion_df["Keyword"]==keyword]
        newSuggestion=set(new_df["Suggestion"].to_list())
        oldSuggestion=set(old_df["Suggestion"].to_list())
        commonSuggestion=list(newSuggestion & oldSuggestion)
        new_Suggestion=list(newSuggestion - oldSuggestion)
         
        for suggest in commonSuggestion:
            suggestionCommon_list.append([dateTimeObj,keyword,suggest])
        for suggest in new_Suggestion:
            suggestionNew_list.append([dateTimeObj,dateTimeObj,keyword,suggest])
    
    #new keywords
    newSuggestion_df = pd.DataFrame(suggestionNew_list, columns=['first_seen','last_seen','Keyword','Suggestion'])
    #shared keywords with date update
    commonSuggestion_df = pd.DataFrame(suggestionCommon_list, columns=['last_seen','Keyword','Suggestion'])
    merge=pd.merge(suggestion_df, commonSuggestion_df, left_on=["Suggestion"], right_on=["Suggestion"], how='left')
    merge = merge.rename(columns={'last_seen_y': 'last_seen',"Keyword_x":"Keyword"})
    merge["last_seen"].fillna(merge["last_seen_x"], inplace=True)
    del merge["last_seen_x"]
    del merge["Keyword_y"]
    
    #merge old results with new results
    frames = [merge, newSuggestion_df]
    keywords_df =  pd.concat(frames, ignore_index=True, sort=False)
    # Save dataframe as a CSV file
    keywords_df['first_seen'] = pd.to_datetime(keywords_df['first_seen'])
    keywords_df = keywords_df.sort_values(by=['first_seen','Keyword'], ascending=[False,False])   
    keywords_df['first_seen']= pd.to_datetime(keywords_df['first_seen'])
    keywords_df['last_seen']= pd.to_datetime(keywords_df['last_seen'])
    keywords_df['is_new'] = (keywords_df['first_seen']== keywords_df['last_seen'])
    keywords_df=keywords_df[['first_seen','last_seen','Keyword','Suggestion','is_new']]
    keywords_df.to_csv('keyword_suggestions.csv', index=False)

# If you use more than 50 seed keywords you should slow down your requests - otherwise google is blocking the script
# If you have thousands of seed keywords use e.g. WAIT_TIME = 1 and MAX_WORKERS = 5
WAIT_TIME = 0.2
MAX_WORKERS = 20
# set the autocomplete language
LANGUAGE = "en"
# set the autocomplete country code - DE, US, TR, GR, etc..
COUNTRY="US"
# Keyword_seed csv file name. One column csv file.
#csv_fileName="keyword_seeds.csv"
CSV_FILE_NAME="keywords.csv"
autocomplete(CSV_FILE_NAME)
#The result will save in keyword_suggestions.csv csv file

Unduh Script Python

Naon anu anjeun pikirkeun?

Situs migunakeun Akismet pikeun ngurangan spam. Diajar kumaha komentar data anjeun diolah.