DOCUMENTATION FOR CHUNKSTRING MODULE

By Stumpy

Chunkstring.pyd is a C-language module created to accelerate processing of some important time-intensive functions. It is mainly used by the MissionObjVar and MLOGReader.

 

Random Info

_version()

returns a float with the current module version.

_versionString()

returns a string with the module version.

 

Helpers for missionobjvar module

numtuple2string(numTuple)

Given a tuple of floats, return the string from which numTuple was created.

string2numtuple(string)

Given string, return a tuple of floats into which that string may be compressed.

 

Functions for parsing ff.log file

The examples in this section assume the following (very edited) ff.log file. It is long and full of stuff we don't (yet) see a need for. But, that's the point: we are trying to parse the log file down to just the useful lines and then grab what we need from them. 
00:00:03.89: FF: CRender: trying to render in 32 bits
00:00:04.20: FF: DepthStencil is D3DFMT_D24S8
00:00:04.65: FF: CResourceTrackerTree: Generating index file Custom_Art_library_characters.dat
00:00:04.68: FF: CResourceTrackerTree: Generating index file Custom_Missions.dat
00:00:04.70: FF: CResourceTrackerTree: Generating index file Custom_Art_library_area_specific.dat
00:00:05.25: FF: ERROR! RPGA attribute bulktp is not mapped to a bitset enum!
00:00:05.25: FF: ERROR! RPGA attribute overheated is not mapped to a bitset enum!
00:00:05.25: FF: ERROR! RPGA attribute metashieldp is not mapped to a bitset enum!
00:01:07.11: FF: animator:(hero_0) setSequence hover
00:01:07.11: FF: animator:(hero_0) setSequence2
00:01:07.11: FF: animator:(hero_0) disableCurrentState
00:01:07.11: FF: animator:(hero_0) CAnimator::stopAnimating: Deactivate Sequence hover_ranged
00:01:07.11: FF: animator:(hero_0) Calling LayerDeactivate for hover_ranged 
00:01:07.11: FF: animator:(hero_0) end disableCurrentState
00:01:07.11: FF: animator:(hero_0) end setSequence2
00:01:07.11: FF: animator:(hero_0) end setSequence hover
00:01:07.12: FF: animator:(hero_0) setSequence fly
00:01:07.12: FF: animator:(hero_0) setSequence2
00:01:07.12: FF: animator:(hero_0) disableCurrentState
00:01:07.12: FF: animator:(hero_0) end disableCurrentState
00:01:07.12: FF: animator:(hero_0) end setSequence2
00:01:07.12: FF: animator:(hero_0) end setSequence fly
00:01:07.12: FF: animator:(hero_0) startSequenceSwap
00:01:07.12: FF: animator:(hero_0) BlendActivate hover -> fly 
00:01:07.12: FF: animator:(hero_0) end startSequenceSwap 0.350000
00:01:07.47: FF: animator:(hero_0) startAnimating fly
00:01:07.47: FF: animator:(hero_0) end startAnimating fly
00:01:07.97: FF: animator:(hero_0) setSequence hover
00:01:07.97: FF: animator:(hero_0) setSequence2
00:01:07.97: FF: animator:(hero_0) disableCurrentState
00:01:07.97: FF: animator:(hero_0) CAnimator::stopAnimating: Deactivate Sequence fly
00:01:07.97: FF: animator:(hero_0) Calling LayerDeactivate for fly 
00:01:07.97: FF: animator:(hero_0) end disableCurrentState
00:01:07.97: FF: animator:(hero_0) end setSequence2
00:01:07.97: FF: animator:(hero_0) end setSequence hover
00:01:07.97: FF: animator:(hero_0) startSequenceSwap
00:01:07.97: FF: animator:(hero_0) BlendActivate fly -> hover 
00:01:07.97: FF: animator:(hero_0) end startSequenceSwap 1.000000
00:56:10.99: FF: destroying font resources
00:56:11.13: FF: Recreating font resources
00:56:11.14: FF: DepthStencil is D3DFMT_D16
00:56:11.14: FF: 16 bit Z, Using W Buffering
00:56:11.54: FF: initializing physics
00:56:22.55: FF: CGameTypeCampaign::postObjectsLoaded(): hero marker not located in mission.dat file
00:56:22.72: FF: ERROR: CRpgSystem::patchRPGABitSet, tombstone has unknown attribute telepathy
00:56:22.72: FF: ERROR: CRpgSystem::patchRPGABitSet, tombstone has unknown attribute nocturnal
00:57:51.63: FF: Recreating font resources
00:58:07.46: FF: activation: >>>>>>>>>add id 75563160, index 152, stamp 1153
00:58:07.47: FF: activation: Creating object testcusto2m library\characters\blackjack\character.nif
00:58:07.48: FF: activation: Activating object testcusto2m
00:58:14.94: FF: freeing physics
00:04:43.41: FF: animator:(thug_with_bat_9) setSequence melee_idle
00:04:43.41: FF: animator:(thug_with_bat_9) Ignoring sequence request, already animating looping melee_idle
00:04:43.43: FF: animator:(thug_with_bat_9) setSequence run
00:04:43.43: FF: animator:(thug_with_bat_9) setSequence2
00:04:43.43: FF: animator:(thug_with_bat_9) disableCurrentState
00:04:43.43: FF: animator:(thug_with_bat_9) stopSequenceSwap
00:04:43.43: FF: animator:(thug_with_bat_9) BlendDeactivate run -> melee_idle 
00:04:43.43: FF: animator:(thug_with_bat_9) end stopSequenceSwap
00:04:43.43: FF: animator:(thug_with_bat_9) end disableCurrentState
00:04:43.43: FF: animator:(thug_with_bat_9) end setSequence2
00:04:43.43: FF: animator:(thug_with_bat_9) end setSequence run
00:04:43.43: FF: animator:(thug_with_bat_9) startSequenceSwap
00:04:43.43: FF: animator:(thug_with_bat_9) BlendActivate melee_idle -> run 
00:04:43.43: FF: animator:(thug_with_bat_9) end startSequenceSwap 0.100000
00:04:43.55: FF: animator:(thug_with_bat_9) startAnimating run
00:04:43.55: FF: animator:(thug_with_bat_9) end startAnimating run
00:04:44.21: FF: animator:(thug_with_bat_9) setSequence melee_idle
00:04:44.21: FF: animator:(thug_with_bat_9) setSequence2
00:04:44.21: FF: animator:(thug_with_bat_9) disableCurrentState
00:04:44.21: FF: animator:(thug_with_bat_9) CAnimator::stopAnimating: Deactivate Sequence run
00:04:44.21: FF: animator:(thug_with_bat_9) Calling LayerDeactivate for run 
00:04:44.21: FF: animator:(thug_with_bat_9) end disableCurrentState
00:04:44.21: FF: animator:(thug_with_bat_9) end setSequence2
00:04:44.21: FF: animator:(thug_with_bat_9) end setSequence melee_idle
00:04:44.21: FF: animator:(thug_with_bat_9) startSequenceSwap
00:04:44.21: FF: animator:(thug_with_bat_9) BlendActivate run -> melee_idle 
00:04:44.21: FF: animator:(thug_with_bat_9) end startSequenceSwap 0.500000
00:04:44.29: FF: animator:(thug_with_bat_9) setSequence melee_idle
00:04:44.29: FF: animator:(thug_with_bat_9) Ignoring sequence request, already animating looping melee_idle
00:58:17.56: FF: activation: >>>>>>>>>add id 131203073, index 1, stamp 2002
00:58:17.56: FF: activation: Creating object _impobj_0 library\area_specific\underground\terrain\3b1_terrain.NIF
00:58:18.15: FF: activation: >>>>>>>>>add id 131268610, index 2, stamp 2003
00:58:18.15: FF: activation: Creating object _impobj_60 library\area_specific\underground\decor\lantern_pole\lantern_pole.nif
00:58:18.15: FF: activation: >>>>>>>>>add id 131334147, index 3, stamp 2004
00:58:18.15: FF: activation: Creating object _impobj_51 library\area_specific\underground\boulders\vb_boulder\vb_boulder.nif
00:58:18.16: FF: activation: >>>>>>>>>add id 131399684, index 4, stamp 2005
00:58:18.16: FF: activation: Creating object intro_shaman library\characters\dark_shaman\character.nif
00:58:18.17: FF: activation: Activating object intro_shaman
00:58:44.58: FF: activation: >>>>>>>>>add id 141033623, index 151, stamp 2152
00:58:44.58: FF: activation: Creating object mobj_0 library\fx\projectile_standin\projectile_standin.nif
00:58:44.58: FF: activation: Activating object mobj_0
00:58:44.67: FF: activation: Activating object _impobj_71
00:58:44.68: FF: activation: CGameObjFactory::removeObjectFromSystem: queing for removal mobj_0
00:58:44.68: FF: activation: De-Activating object mobj_0
00:58:44.75: FF: activation: CGameObjFactory::removeObject: removing mobj_0
00:58:44.75: FF: activation: invalidate id 141033623, index 151, stamp 2152
00:08:50.62: FF: Movement:(minute_man) setting destination: (-898.909302,902.995728,-2.058727)
00:02:29.48: FF: Movement:(minute_man) setting destination radius: 0.000000

I will show some examples below intended to illustrate how the chunkstring functions can be used to grab just the few lines of interest from the above ff.log segments. In each example, I looped the function in question over the whole set of above lines and I show which lines produced a non-None result and what those results were (generally a non-empty tuple or a zero/one value for use as a boolean).

I've grouped the functions by whether they are specific to one kind of log line or of more general nature.

 

General

MLOG_GetModuleName(line)

Given line from ff.log, determine if this is a line of normal log output (as opposed to script.log output before the script.log was initialized) and, if so, return the MLOG module that sent the line. If not, returns None.

Example:
There were too many lines to show here, since nearly all the above lines had some module output. The modules detected were:
'CRender', 'CResourceTrackerTree', 'CGameTypeCampaign', 'ERROR', 'animator', 'animator', and 'Movement'

ParseByModuleAndKeyword(line,module,keyword,getToken[,exclude])

Given a line from ff.log, determine if this is a line from the given module containing the given keyword (any string after the object name) in it. If so, find the (space-delimited) token located getToken tokens from the keyword and return a tuple with the object name (if it is found in parenthesis after the module name, as per the animator and Movement modules) and the token. If not, or if optional string exclude is found in line, returns None. GetToken can be negative (indicating tokens before the keyword).
Example 1:
Running ParseByModuleAndKeyword(line,'Movement','setting destination',1) on all the lines in the contrived ff.log triggered on the following lines and returned the tuple shown below each line. 
00:08:50.62: FF: Movement:(minute_man) setting destination: (-898.909302,902.995728,-2.058727)
	--> ('minute_man', '(-898.909302,902.995728,-2.058727)')
00:02:29.48: FF: Movement:(minute_man) setting destination radius: 0.000000
	--> ('minute_man', 'radius:')
Example 2:
But what if we didn't want to trigger on the second line because, maybe, we just want lines with the destination tuple in them. We could specify the optional exclude paramater. Thus, ParseByModuleAndKeyword(line,'Movement','setting destination',1,'radius') would give us 
00:08:50.62: FF: Movement:(minute_man) setting destination: (-898.909302,902.995728,-2.058727)
	--> ('minute_man', '(-898.909302,902.995728,-2.058727)')
Example 3:
Or, we might just want the radius lines be calling just ParseByModuleAndKeyword(line,'Movement','setting destination',2) or ParseByModuleAndKeyword(line,'Movement','setting destination radius ',1). The first will exclude the destination only lines because those lines have only one token after 'setting destination' and the second explicitly includes ' radius' at the end of the keyword string.

IsFFXLogLine(line)

Given a line from ff.log, determine if this is a line ffx.parseLogFile() should examine, if so, return 1. If not, returns 0.
Example:
The lines that might be of interest in the (old) parseLogFile() method of initializing FFX attributes could be gotten with >IsFFXLogLine(line). 
00:56:11.54: FF: initializing physics
	--> 1
00:56:22.72: FF: ERROR: CRpgSystem::patchRPGABitSet, tombstone has unknown attribute telepathy
	--> 1
00:56:22.72: FF: ERROR: CRpgSystem::patchRPGABitSet, tombstone has unknown attribute nocturnal
	--> 1

 

Animator

ParseAnimatorForSetsequence(line[,exclude])

Given a line from ff.log, determine if this is an animator setSequence line and, if so, return a two-tuple of the object name and the animation name. If not, or if optional string exclude is found in line, returns None.
Example:
The lines that we use in flight, flailing, etc. detection trigger this function. Of the above log lines, ParseAnimatorForSetsequence(line) finds: 
00:01:07.11: FF: animator:(hero_0) setSequence hover
	--> ('hero_0', 'hover')
00:01:07.12: FF: animator:(hero_0) setSequence fly
	--> ('hero_0', 'fly')
00:01:07.97: FF: animator:(hero_0) setSequence hover
	--> ('hero_0', 'hover')
00:04:43.41: FF: animator:(thug_with_bat_9) setSequence melee_idle
	--> ('thug_with_bat_9', 'melee_idle')
00:04:43.43: FF: animator:(thug_with_bat_9) setSequence run
	--> ('thug_with_bat_9', 'run')
00:04:44.21: FF: animator:(thug_with_bat_9) setSequence melee_idle
	--> ('thug_with_bat_9', 'melee_idle')
00:04:44.29: FF: animator:(thug_with_bat_9) setSequence melee_idle
	--> ('thug_with_bat_9', 'melee_idle')

ParseAnimations: ParseAnimations(line)

Given a line from ff.log, determine if this is an animator setSequence line and, if so, return a two-tuple of the object name and the animation name. If not, returns None.

(This function is essentially the same as ParseAnimatorForSetsequence() and is provided for compatibility.)

ParseAnimatorByKeyword(line,keyword,getToken[,exclude])

Given a line from ff.log, determine if this is an animator line with the given keyword (any string after the object name) in it. If so, find the (space-delimited) token located getToken tokens from the keyword and return a tuple with the object name and the token. GetToken can be negative (indicating tokens before the keyword). If not, or if optional string exclude is found in line, returns None.
Example 1:
The lines that might be of interest in, for example, idle detection, can be found calling ParseAnimatorByKeyword(line,'startAnimating',1). 
00:01:07.47: FF: animator:(hero_0) startAnimating fly
	--> ('hero_0', 'fly')
00:01:07.47: FF: animator:(hero_0) end startAnimating fly
	--> ('hero_0', 'fly')
00:04:43.55: FF: animator:(thug_with_bat_9) startAnimating run
	--> ('thug_with_bat_9', 'run')
00:04:43.55: FF: animator:(thug_with_bat_9) end startAnimating run
	--> ('thug_with_bat_9', 'run')
Example 2:
But that actually gets too many lines, since we don't really need the 'end startAnimating' lines. By calling ParseAnimatorByKeyword(line,'startAnimating',1,'end startAnimating') we get 
00:01:07.47: FF: animator:(hero_0) startAnimating fly
	--> ('hero_0', 'fly')
00:04:43.55: FF: animator:(thug_with_bat_9) startAnimating run
	--> ('thug_with_bat_9', 'run')
Note that it's safer not to specify just 'end' as the exclude parameter because that would ignore any lines with 'end' in their animation names.

 

Activation

ParseActivationForNewObject(line)

Given a line from ff.log, determine if this is an activation module line creating an object, if so, return a tuple with the object name and NIF path. If not, returns None.
Example:
This could be very handy for setting up spawn sinks. By calling ParseActivationForNewObject(line) we get 
00:58:07.47: FF: activation: Creating object testcusto2m library\characters\blackjack\character.nif
	--> ('testcusto2m', 'library\\characters\\blackjack\\character.nif')
00:58:17.56: FF: activation: Creating object _impobj_0 library\area_specific\underground\terrain\3b1_terrain.NIF
	--> ('_impobj_0', 'library\\area_specific\\underground\\terrain\\3b1_terrain.NIF')
00:58:18.15: FF: activation: Creating object _impobj_60 library\area_specific\underground\decor\lantern_pole\lantern_pole.nif
	--> ('_impobj_60', 'library\\area_specific\\underground\\decor\\lantern_pole\\lantern_pole.nif')
00:58:18.15: FF: activation: Creating object _impobj_51 library\area_specific\underground\boulders\vb_boulder\vb_boulder.nif
	--> ('_impobj_51', 'library\\area_specific\\underground\\boulders\\vb_boulder\\vb_boulder.nif')
00:58:18.16: FF: activation: Creating object intro_shaman library\characters\dark_shaman\character.nif
	--> ('intro_shaman', 'library\\characters\\dark_shaman\\character.nif')
00:58:44.58: FF: activation: Creating object mobj_0 library\fx\projectile_standin\projectile_standin.nif
	--> ('mobj_0', 'library\\fx\\projectile_standin\\projectile_standin.nif')