Need Some Help With Python.

Talk about computers, hardware, applications, and consumer electronics.
User avatar
Moderator
Moderator
Posts: 1402
Joined: 2008.10.01 (01:36)
NUMA Profile: http://nmaps.net/user/therealone

Postby TheRealOne » 2010.01.04 (01:06)

OK, so if you haven't read the thread I posted in Highscores I am trying to port over runs from the metanet servers to the nreality server, but it a very tedious process and I am wondering if I could use Nhigh to my advantage. The hard part of importing these runs is getting the demo data for each run. Nhigh gathers the demo data from any run in the top-20 you ask for, however it then changes the demo data in to a text format for ease of reading, but I am wondering if I could edit the nhigh python files so that it does not do this last step, and instead will only display the demo data.

Here are the section of python code that are applicable for the demo analysis function.
Code 1

Code: Select all

    def demoAnalysis(self):
        SEL_TOP20, SEL_DEMO = 'top20', 'demo'
        frames = [
            AskDlgSelectionSubFrame(
                [
                    AskDlgHorizSubFrame([
                        IntField('episode','Episode',minValue=0,maxValue=99, width=3),
                        IntField('level','Level',minValue=0,maxValue=4, width=3),
                        IntField('rank','Rank',minValue=0,maxValue=19, width=3),
                    ]),
                    MultiLineField('demo','Demo data'),
                ],
                ['Online Top-20 Score', 'Demo Data'],
                [SEL_TOP20, SEL_DEMO]),
        ]
        dlg = AskDialog(self.root, 'Demo Analysis',
            'Description: Analyzes a demo and shows\nwhich keys were pressed for each frame',
            frames)
        res = dlg.run()
        if not res:
            return
        if res.selection==SEL_TOP20:
            player, score, demo = downloadReplay(res.episode, res.level, res.rank)
            title = 'Demo analysis for %s (%s rank) - [%s, %s]' % \
                    (self.dispLevel(res.episode, res.level),
                     getOrdinal(res.rank), self.dispPlayer(player),
                     self.dispTime(score*FRAME_TIME))
        elif res.selection==SEL_DEMO:
            demo = res.demo
            title = 'Demo analysis'
        else:
            return

        arr = parseAndCompactReplay(demo)
        self.setText(title+'\n\n')
        for frame, key, num in arr:
            self.addLine('%03d. %-7s - %d frame%s' %
                (frame, getReplayKeyName(key), num, getPlural(num)))
[/spoiler]
Code 2

Code: Select all

##### replay download/analysis #####

def getReplayKey(ep, lvl, rank):
    if ep<0 or ep>=NUM_EPISODES or lvl<0 or lvl>4 or rank<0 or rank>19:
        return None
    url = 'http://www.harveycartel.org/metanet/n/data13/get_topscores_query_jg.php'
    postdata = 'episode%5Fnumber=' + str(ep)
    allscores = openURL(url, postdata).replace('\r','')
    searchstr = r'&%dpkey%d=(\d+)' % (lvl, rank)
    m=re.search(searchstr, allscores)
    if not m:
        return None
    return int(m.group(1))

def downloadReplayByPKey(pkey):
    url = 'http://www.harveycartel.org/metanet/n/data13/get_lv_demo.php'
    postdata = 'pk='+str(pkey)
    return openURL(url, postdata).replace('\r','')

def downloadReplay(ep, lvl, rank):
    'Returns tuple: (player,score,demo)'
    try:
        pkey = getReplayKey(ep, lvl, rank)
        if pkey is None:
            raise NHighError('Unable to download replay - replay key not found')

        alldata = downloadReplayByPKey(pkey)
        m_name=re.search('&name=([^&]+)',alldata)
        m_score=re.search('&score=([^&]+)',alldata)
        m_demo=re.search('&demo=([^&]+)',alldata)
        try:
            player = m_name.group(1)
            demo = m_demo.group(1)
            score = int(m_score.group(1))
        except (ValueError, AttributeError):
            raise NHighError('Unable to download replay - received invalid data')

        return (player, score, demo)        
        
    except IOError:
        raise NHighError('Error downloading replay data')
 
def getReplayKeyName(key):
    ret = ''
    if key&1: ret += 'L'
    if key&2: ret += 'R'
    if key&4: ret += 'J'
    if key==0: ret = 'Nothing'
    return ret

def parseReplay(data):
    '''Parse the replay and return array of numbers (1 per frame).
    The numbers can be passed to getReplayKeyName.'''

    if data and data[0]=='$':
        #skip level data
        m = re.match(r'^\$[^#]*#[^#]*#[^#]*#[^#]*#([^#]*)',data)
        if not m:
            raise NHighError('Invalid demo data')
        data = m.group(1)

    m=re.match(r'^(\d+):((\d+\|)*\d+)$', data)
    if not m:
        raise NHighError('Invalid demo data')
    frames = int(m.group(1))
    keystring = m.group(2)
    nums = [int(x) for x in re.findall(r'\d+',keystring)]
    ret = []
    for num in nums:
        numbits = min(frames, 7)
        for bit in xrange(numbits):
            ret.append(num & 0xF)
            num >>= 4
            
        frames -= numbits
        if frames==0 and num!=0:
            #framecount is lower than number of provided frames
            return ret
    return ret

    
def parseAndCompactReplay(data):
    '''Parse the replay, and then find sequences of unchanged keys.
    Return array of tuples (frame_num, key_num, # of occurences)
    The key numbers can be passed to getReplayKeyName.'''

    ret = []
    keys=parseReplay(data)
    if not keys:
        return ret
    keys.append(-1)
    lastkey=-1
    count=0
    frame=0
    for key in keys:
        key = key&7
        if key==lastkey:
            count+=1
        else:
            if lastkey!=-1:
                ret.append((frame-count, lastkey, count))
            lastkey = key
            count = 1
        frame += 1
    return ret
   
[/spoiler]

The first set of code I got from the Nhigh.pyw file which you run to use nhigh. The second set I got from the nhighlib.py file.

Now I am almost complete illiterate when it comes to python, so any help would be great. Again I am wondering if I could edit one of these files, or if someone could tell me another quick way like running something in python shell, so that I could get the demo data for these runs quickly.

Also you can download Nhigh here.
<@Izzy> Holy balls, sweet run.
<@gloomp> Holy sweet, balls run.
<@Izzy> Sweet, balls run holy.
<@gloomp> Run sweet, balls holy.
<@Izzy> Sweet run, ballsy hole.
<@gloomp> All's sweet, holeb run.
<@Izzy> Ballsy nun, sweet mole.
Image

<@Kool> bro no joke, I saw the sexiest swedish chick giving herself anal on one of those pop-up alarm-clocks at my uncle's house

User avatar
Global Mod
Global Mod
Posts: 1416
Joined: 2008.09.26 (05:35)
NUMA Profile: http://nmaps.net/user/scythe33
MBTI Type: ENTP
Location: 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0

Postby scythe » 2010.01.04 (01:46)

It doesn't matter what the demo analysis and formatting function does, you just need to comment out the line where it's called. From what I can see you didn't post it.
As soon as we wish to be happier, we are no longer happy.

User avatar
Moderator
Moderator
Posts: 1402
Joined: 2008.10.01 (01:36)
NUMA Profile: http://nmaps.net/user/therealone

Postby TheRealOne » 2010.01.04 (04:28)

scythe33 wrote:It doesn't matter what the demo analysis and formatting function does, you just need to comment out the line where it's called. From what I can see you didn't post it.
Ok, I don't really understand what you mean by "you just need to comment out the line where it's called," but I am assuming you mean I just either need to delete the demo analysis part or put some sort of python notation called "comment" that skips that process. I am pretty sure I posted everything that Nhigh does to get the demo data.

In the second set of code I am pretty sure everything up to "def getReplayKeyName(key):" is getting the demo data and then from that and onward is when it changed the demo data in to text. But I tried deleting everything after and including "def getReplayKeyName(key):" but Nhigh then didn't do anything when I tried the function. I think that if i could make it display what it has at "return (player, score, demo)" that point it would work, but that is my question I do not know how to do that.
<@Izzy> Holy balls, sweet run.
<@gloomp> Holy sweet, balls run.
<@Izzy> Sweet, balls run holy.
<@gloomp> Run sweet, balls holy.
<@Izzy> Sweet run, ballsy hole.
<@gloomp> All's sweet, holeb run.
<@Izzy> Ballsy nun, sweet mole.
Image

<@Kool> bro no joke, I saw the sexiest swedish chick giving herself anal on one of those pop-up alarm-clocks at my uncle's house

Plus (Size) Member
Posts: 42
Joined: 2008.09.27 (02:56)

Postby taaveti » 2010.01.05 (03:34)

Without more context I can't be sure, but it looks like what you're trying to do is to replace (from the bottom of "Code 1")

Code: Select all

        arr = parseAndCompactReplay(demo)
        self.setText(title+'\n\n')
        for frame, key, num in arr:
            self.addLine('%03d. %-7s - %d frame%s' %
                (frame, getReplayKeyName(key), num, getPlural(num)))
with

Code: Select all

        self.setText(title+'\n\n')
        self.addLine(demo)

User avatar
Moderator
Moderator
Posts: 1402
Joined: 2008.10.01 (01:36)
NUMA Profile: http://nmaps.net/user/therealone

Postby TheRealOne » 2010.01.05 (09:25)

taaveti wrote:Without more context I can't be sure, but it looks like what you're trying to do is to replace (from the bottom of "Code 1")

Code: Select all

        arr = parseAndCompactReplay(demo)
        self.setText(title+'\n\n')
        for frame, key, num in arr:
            self.addLine('%03d. %-7s - %d frame%s' %
                (frame, getReplayKeyName(key), num, getPlural(num)))
with

Code: Select all

        self.setText(title+'\n\n')
        self.addLine(demo)
Sir, you are a genius! I can not thank you enough for this. It worked perfectly.
<@Izzy> Holy balls, sweet run.
<@gloomp> Holy sweet, balls run.
<@Izzy> Sweet, balls run holy.
<@gloomp> Run sweet, balls holy.
<@Izzy> Sweet run, ballsy hole.
<@gloomp> All's sweet, holeb run.
<@Izzy> Ballsy nun, sweet mole.
Image

<@Kool> bro no joke, I saw the sexiest swedish chick giving herself anal on one of those pop-up alarm-clocks at my uncle's house

User avatar
Queen of All Spiders
Posts: 4263
Joined: 2008.09.29 (03:54)
NUMA Profile: http://www.freeWoWgold.edu
MBTI Type: ENFP
Location: Quebec, Canada!

Postby SlappyMcGee » 2010.01.05 (20:59)

taaveti wrote:Without more context I can't be sure, but it looks like what you're trying to do is to replace (from the bottom of "Code 1")

Code: Select all

        arr = parseAndCompactReplay(demo)
        self.setText(title+'\n\n')
        for frame, key, num in arr:
            self.addLine('%03d. %-7s - %d frame%s' %
                (frame, getReplayKeyName(key), num, getPlural(num)))
with

Code: Select all

        self.setText(title+'\n\n')
        self.addLine(demo)
Awesome work, dude.
Loathes

Plus (Size) Member
Posts: 42
Joined: 2008.09.27 (02:56)

Postby taaveti » 2010.01.06 (03:55)

Incendentally, it would probably make your life easier if you wrote the demo data to a file for further processing, rather than to a text box for manual copy/paste. Replace " self.addLine(demo)" with

Code: Select all

        fd = open('/some/file/path/here', 'w')
        fd.write(demo)
        fd.close()
possibly referencing player and/or some other uniquifier in the filename so you don't have to process after each download.

User avatar
Moderator
Moderator
Posts: 1402
Joined: 2008.10.01 (01:36)
NUMA Profile: http://nmaps.net/user/therealone

Postby TheRealOne » 2010.01.12 (06:39)

taaveti wrote:Incendentally, it would probably make your life easier if you wrote the demo data to a file for further processing, rather than to a text box for manual copy/paste. Replace " self.addLine(demo)" with

Code: Select all

        fd = open('/some/file/path/here', 'w')
        fd.write(demo)
        fd.close()
possibly referencing player and/or some other uniquifier in the filename so you don't have to process after each download.
Ok this sounds like it could be really helpful. Could I do this with an excel file? and how would the program know where to paste the data in the excel file? like to which cells will it paste?
<@Izzy> Holy balls, sweet run.
<@gloomp> Holy sweet, balls run.
<@Izzy> Sweet, balls run holy.
<@gloomp> Run sweet, balls holy.
<@Izzy> Sweet run, ballsy hole.
<@gloomp> All's sweet, holeb run.
<@Izzy> Ballsy nun, sweet mole.
Image

<@Kool> bro no joke, I saw the sexiest swedish chick giving herself anal on one of those pop-up alarm-clocks at my uncle's house

Plus (Size) Member
Posts: 42
Joined: 2008.09.27 (02:56)

Postby taaveti » 2010.01.13 (03:54)

The above will just write it to a file (i.e. plain text). While it would be possible to paste into a spreadsheet, it would require either internal knowledge of excel or synthesizing click/keyboard events, which seems like way more effort than would be reasonable. If you absolutely have to have it in excel, my suggestion would be to put all the demos in one file (replace 'w' with 'a' in the call to file.open and "demo" with "demo+'\n'" in the call to fd.write) and import it from within excel.

Raigan and the Horse-Woman
Raigan and the Horse-Woman
Posts: 182
Joined: 2008.09.27 (02:14)
NUMA Profile: www.nmaps.net/user/sidke
Steam: www.steamcommunity.com/id/shagdish
Location: ⑨ 
Contact:

Postby sidke » 2010.01.13 (05:38)

TheRealOne wrote:Ok this sounds like it could be really helpful. Could I do this with an excel file? and how would the program know where to paste the data in the excel file? like to which cells will it paste?
I haven't used this, but a quick google search produced a strangely appropriate group of modules: http://www.python-excel.org/
spoiler

辻菜摘が好きじゃー ヽ(´ー`)ノ sig by peking duck


User avatar
Global Mod
Global Mod
Posts: 1416
Joined: 2008.09.26 (05:35)
NUMA Profile: http://nmaps.net/user/scythe33
MBTI Type: ENTP
Location: 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0

Postby scythe » 2010.01.13 (21:54)

You could just write it to a CSV. Excel can open those just fine, and comma-separated values ought to be sufficient for level data.
As soon as we wish to be happier, we are no longer happy.


Who is online

Users browsing this forum: No registered users and 13 guests