Build an audio fragments merger with gui for qlive App courses

2019年09月10日 1021Browse 4Like 0Comments

Scenario

The qlive audio courses cannot be played or downloaded directly from its website, although the customer service staff promise they can be downloaded from the APP once you make the payment. However, what they promise is not a promise, the APP just buffers your course files into your phone, and the files are hidden and splitted to many small pieces in device's sdcard and can only be played from the APP, each time before you find your course in the APP you have to login and do several clicks of their annoying ads.

We hope to get a clean and single mp3 media file for each chapter of the course when we don't have a WIFI access, so that we can listen to the paid course offline and without device/App restrictions.

Analysis

  1. Install qlive App into a VirtualXposed Container
  2. Use HttpCanary to capture the buffering stream packets and find that, each chapter's media(audio) content:
    • is not played from a single media file like xxx.mp3
    • is divided into many fragments files (*.m4a, *.mp4, *.aac format) and NOT SORTED BY SEQUENCE, no obvious file-name rules as well
  3. Analyze the App directories and files:
    • audio fragments are buffered into a hidden directory: /sdcard/qlive/.voice/ by chapters
    • fragments' play order is stored in a REALM database file /data/data/com.thinkwu.live/files/default.realm
    • buffered media fragments are not encrypted, we do not need to decrypt them
  4. Use Realm Studio to export the database file to a json format file, and we will find many key/secret information: the relation of the fragments file and chapter topics, the paly order of the fragements etc.

Solution

  1. Use python to analyze/parse the json file to:
    • get course chapters' detail information
    • get correct sequence of all fragments when playing each chapter/topic
    • batch rename all unsorted fragment files for each chapter
    • call ffmpeg to merge all buffered fragment files into a single playable mp3 file
  2. Implementation:
    • Imported key modules:
         import tkinter						# tkinter/ttk gui
         from tkinter import ttk     			# treeview control
         from tkinter import messagebox
         from tkinter.filedialog import askdirectory
         
         import json							# load structures in json to python dicts
         import re							# process unusual charaters of filename string
         
         import os
         import subprocess					# call ffmpeg tools to merge fragments
         import time
         
    • tkinter gui initialization:
         def __init__(self):   
             # main window initilisation
             self.root = tkinter.Tk()
             #self.root.state("zoomed")      
             self.root.geometry("800x400+220+150")   
             self.root.resizable(0,0)  
             # self.root.wm_attributes('-topmost', 1)      
             self.root.title("qlLive Audio Exporter v0.01")   
         
             # elements for select root path 
             self.label = tkinter.Label(self.root, text="Set course files root direcotry:", anchor="w")
             self.path_entry = tkinter.Entry(self.root, textvariable = self.root_path, width=100)
             self.path_entry.insert(0, self.root_path) 
             self.setpath_button = tkinter.Button(self.root, command = self.select_path, text = "Browse")
         
             # use a treeview to create a table/list for course topics
             self.tree = ttk.Treeview(self.root, show="headings")     
             self.tree["columns"]=("1","2","3", "4", "5", "6", "7")
             self.tree.column("1",width=5, anchor='center')        
             self.tree.column("2",width= 80)
      
      	   self.tree.column("3",width=200)
      	   self.tree.column("4",width=50, anchor='center')
      	   self.tree.column("5",width=50, anchor='center')
      	   self.tree.column("6",width=50, anchor='center')        
      	   self.tree.column("7",width=50, anchor='center')
      
      	   self.tree.heading("1",text="No.")  
      	   self.tree.heading("2",text="Topic ID")
      	   self.tree.heading("3",text="Topic Name")
      	   self.tree.heading("4",text="Slice Nubmer")
      	   self.tree.heading("5",text="Downloaded")        
      	   self.tree.heading("6",text="Renamed")
      	   self.tree.heading("7",text="Merge Status")   
      
      	   self.tree.pack()   
      	   self.analyse_button = tkinter.Button(self.root, command = self.analyse_topics, text = "Parse Course")        
      	   self.rename_button = tkinter.Button(self.root, command = self.rename_one_topic, text = "Rename one chapter's fragments")
      	   self.rename_all_button = tkinter.Button(self.root, command = self.rename_all_topics, text = "Rename All chapter's fragments")
      	   self.merge_button = tkinter.Button(self.root, command = self.merge_one_topic, text = "Export selected Topic")
      	   self.merge_all_button = tkinter.Button(self.root, command = self.merge_all_topics, text = "Export all topics")
      
    • tkinter gui elements alignments:
         def gui_arrange(self):        
             self.label.place(x=5, y=5, height=25)
             self.path_entry.place(x=180, y=5, width=550, height=25)
             self.setpath_button.place(x=735, y=5, width=55, height=25)
             self.tree.place(x=10, y=35, width=780, height=315)
      
             self.analyse_button.place(x=30, y=360, width=90)
      
             self.rename_button.place(x=135, y=360, width=185)        
             self.rename_all_button.place(x=330, y=360, width=175) 
      
             self.merge_button.place(x=520, y=360, width=125)                
             self.merge_all_button.place(x=660, y=360, width=110)
         
  • Other key codes: json file process, command file call, etc.
       try:            
           with open(f_realm_json,'r',encoding='UTF-8') as down_json:
               db_dict = json.load(down_json)
               topics = db_dict["DownloadTopicRealmModel"]   
               ……
      
       # call ffmpeg tool to build a merge/export command line
       ffmpeg_cmd = ('ffmpeg -loglevel panic -y -i "concat:%s" -b:a 128k %s' % ('|'.join(topic_voice_list_New), self.output_path + '/' + topic_name +".mp3"))                    
       with open(self.root_path + '/' + topic_id + '/' +'merge.bat', 'w') as f:    
           f.write(ffmpeg_cmd)     # write merge/export command line into a file
       f.close()
       
       exec_ffmpeg_cmd = self.root_path + '/' + topic_id + '/' + 'merge.bat'  
       os.chdir(self.root_path + '/' + topic_id)  
      
       # excute the bat file to call the constructed export command 
       p = subprocess.Popen(exec_ffmpeg_cmd)      
       p.wait()        # ignore to use subprocess.call
       if p.returncode == 0:      
           self.tree.set(topics_items_list[y], column=7, value="Done") 
       
    1. Build an executable program
      > pip3 install pyinstaller
      > pyinstaller -w qlLive_audio_exporter.py

Output

Sunflower

Stay hungry stay foolish

Comments