 |
| Register | Login | |||||
 |
Main
| Memberlist
| Active users
| ACS
| Commons
| Calendar
| Online users Ranks | FAQ | Color Chart | Photo album | IRC Chat |
|
| | ||
|
| Register | Login | |||||
|
Main
| Memberlist
| Active users
| ACS
| Commons
| Calendar
| Online users Ranks | FAQ | Color Chart | Photo album | IRC Chat |
|
| | ||
| 0 user currently in Programming. | 3 guests |
| Acmlm's Board - I2 Archive - Programming - This is never going to get fixed!! |
 |  |  |
| Add to favorites | "RSS" Feed | Next newer thread | Next older thread |
| User | Post | ||
|
fatchicken Newcomer Level: 2 Posts: 1/1 EXP: 14 For next: 32 Since: 03-24-05 Since last post: 223 days Last activity: 223 days |
| ||
| I am trying to get a functional parser to move fields of an original log file (a text file selected through the FileListBox control) to make arrays out of each of the members of data. Here are some of the parts of the original TXT file: The data continues like this for 12000 lines. I want to make arrays from these numbers. Each array will be called according to it's topmost label. For example, everything in the set X1_accel are the numbers 2038, 2032 ... etc. so the array in this category would be: X1_accel(1) = 2038 X1_accel(2) = 2032 ... X1_accel(12000) = (the 12000'th number) likewise another array would be: Y1_accel(1) = 2095 Y1_accel(2) = 2093 ... all the way to Y1_accel(12000) Count,X1_accel,X10_accel,Y1_accel,Y10_accel,Z1_accel,Z10_accel,X_gyro,Y_gyro,Z_gyro,Load1,Load2,Load 3,Load4 1,2038,2095,1983,2023,2816,2103,2037,2018,2025,641,1034,978,512 2,2032,2093,1985,2022,2810,2102,2035,2016,2023,643,1037,981,514 Here is my code... It runs incredibly slow when I do this. I want to be able to work with the numbers in the arrays so I can take data in and graph it using a graphing toolkit. This is hard. Anyone who can help me at all on this problem is greatly appreciated. ------------------------------------------------------------------ Public Function FilterLength() 'On Error GoTo RealErrorHandler ' ******************************************************************************************** ' Global Variables ' ' NumOfSamples long (max less than 15000) ' NumOfSensors integer (1 to 16 is valid range) ' SensorInfo(NumofSensors,16) as string ' RawSensorData(NumOfSensors,NumofSamples) as double ' For sensorcounter = 1 to NumofSensors ' Read the file header ' Do some limit checking on the input header ' REad rest of array data ' Do some checking on array data (# of sensors matches commas, number of samples matches the actual data in file) ' Does it end with the string "end" ' How to indicate VB errors to user via message box, and have clean termination of program (not crash) ' How to indicate logic errors (45r6 instead of 4536) ' ******************************************************************************************** Dim iTest As Integer Dim debugLevel As Integer Dim objInput As Object ' Object for Input Dim objFSO As Object ' Object FileSystemObject Dim objOutput As Object ' Object TOBE the OUTPUT_FILE constant Dim sampleCount As Long ' modGlobals: { Global numOfSamples as long } Dim strMsgBox As String ' When the process is complete Dim numOfSensors As Integer ' Number of sensors in the array Dim intLineLength As Integer ' To find length of each line Dim lngWhereIsComma As Long ' To find the commas in the log Dim bWrite As Boolean ' Boolean because will be T/F Dim Sensor(15000, 13) As Double Dim lngValue As String ' The sample count Dim strCurrentLine As String ' Parse the current line to a string Dim sensorWriteLine As String ' For debugging and printing LineByLine Dim numTemp As Integer ' Temporary Integer Dim strMsgBoxError As String ' Error in RealErrorHandler Dim myCurrentPath As String ' The path in fileSystemBox modal Dim prbLCV As Integer ' The Loop Control Variable for progress bar Dim bCanIDoEvents As Boolean ' T/F for Event handlers Dim INPUT_FILE As String ' The file being read Dim OUTPUT_FILE As String ' The file being sent as [original] + "_LOG" + ".txt" Dim logDTNStamp As String Dim msgBoxNoFileSelected As String Dim timeRightNow As DateTime '******************************************************************************************** INPUT_FILE = ddlFile.FileName OUTPUT_FILE = (INPUT_FILE & "_LOG#" & logDTNStamp & ".TXT") 'TODO: Fix RunTime errors with this problem. Probable causes: The filename already exists. 'VB likes to have constants for the filename, but I want to use a dynamic variable so that 'any TXT file can be selected. ' Set up our FSO object and open the input file Set objFSO = CreateObject("scripting.FileSystemObject") Set objInput = objFSO.OpenTextFile(INPUT_FILE, 1) If chkDebug.Value = vbChecked Then debugLevel = 1 Else debugLevel = 0 End If If INPUT_FILE = "" Then msgBoxNoFileSelected = MsgBox("Please select a file.", vbOKOnly + vbCritical, "No File Selected") '******************************************************************************************** 'For Debugging the program, this will create a file for testing the parsed array If debugLevel = 1 Then Set objOutput = objFSO.CreateTextFile(OUTPUT_FILE, True) Else End If '******************************************************************************************** Let ddlFile.Path = myCurrentPath 'User feedback captions when the loop is started lblTimEnd.Caption = "": lblTimStart.Caption = "": lblTeller.Caption = "" lblTimStart.Caption = DateTime.Now 'Record the start of this procedure before loop lblTeller.Caption = "Please Wait...": lblTeller.ForeColor = &HFF& 'Red 'Create an indicator to tell us if we've hit the part of the file we want to write from bWrite = False sampleCount = 0 'Begin parsing the input file Do While Not objInput.AtEndOfStream DoEvents iTest = iTest + 1 'For debugging purposes strCurrentLine = objInput.ReadLine 'Read the current line If strCurrentLine = "end" Then GoTo ProcessDone If bWrite = True Then 'Write the current line of data to the logfile For numOfSensors = 1 To 13 'Assign inputs to array variables lngWhereIsComma = InStr(strCurrentLine, ",") Sensor(sampleCount, numOfSensors) = Left(strCurrentLine, (lngWhereIsComma - 1)) intLineLength = Len(strCurrentLine) strCurrentLine = Right(strCurrentLine, (intLineLength - lngWhereIsComma)) Next If debugLevel = 1 Then 'Write the output file here For numOfSensors = 1 To 12 'TODO: Do not write CR-LF sensorWriteLine = Sensor(sampleCount, numOfSensors) & "," objOutput.Write sensorWriteLine 'Use WRITE not Writeline Next objOutput.Write Sensor(sampleCount, 13) sampleCount = sampleCount + 1 'Increment the Counter lblSampleCount.Caption = sampleCount prbProg.Value = sampleCount 'Increment the ProgressBar End If End If 'Check to see if we've hit either bound of the data we want If Left(strCurrentLine, 5) = "Count" Then bWrite = True ElseIf strCurrentLine = "end" Then bWrite = False End If Loop ProcessDone: 'For debugging purposes lblTimEnd.Caption = DateTime.Now 'Record the start of this procedure after loop lblTeller.Caption = "Process Complete. The variables have been created.": lblTeller.ForeColor = &HFF0000 'Blue cmdRun.Enabled = True prbProg.Value = 0 'Reset to 0 so when progress runs again it has a starting point GoTo SkipOverErrorHandler 'TODO: Close files. RealErrorHandler: 'Displays whatever runtime error occurs without halting the program strMsgBoxError = MsgBox(Err.Description + " in " + Err.Source, vbOKOnly + vbCritical, "Syntax or critical error") SkipOverErrorHandler: End Function |
|||
Parasyte Bullet Bill Level: 35  Posts: 377/514 EXP: 267348 For next: 12588 Since: 05-25-04 Since last post: 104 days Last activity: 32 days |
| ||
| Visual Basic is slow, as it is. Reading files will make it invariably slower. Creating arrays with 12000 elements from parsing a file will be yet another speed hit. And as you can see, the method you are using will cause more problems, still. I would suggest writing a seperate program to create the arrays and save them into (a single or multiple) binary file(s). Then rewrite this program to simply read the arrays from the binary. It will be quite a bit faster to do this. Alternatively, you could just move your txt parsing code into an "init" routine that runs only once, each time the program is started, and uses the aforementioned binary files. But from what I can tell, this is what you want to avoid? Another option would be rewriting the txt format to help optimize the load time. Yet another option would be rewriting the txt format and using a more viable language. (Though each of these suggestions depend on the possibilities that you may not be able to do so. For example, a school project for a Visual Basic class.) |
|||
|
HyperLamer <||bass> and this was the soloution i thought of that was guarinteed to piss off the greatest amount of people Sesshomaru Tamaranian Level: 118 Posts: 3881/8210 EXP: 18171887 For next: 211027 Since: 03-15-04 From: Canada, w00t! LOL FAD Since last post: 2 hours Last activity: 2 hours |
| ||
I hate to say this - and indeed if you'd said it to me before I really got into C, I'd probably have ignored you - but you'd be better off doing a simple C version, maybe creating output that would be easier for a VB program to handle. C is more difficult than VB, yes, but this isn't really a difficult task. All you'd really need to use is fopen(), fscanf() and fclose().  You'll find that C is wickedly fast compared to VB when it comes to file I/O and dealing with large arrays.And if you can't or won't do that, rewriting the file format a bit might help. Pad the digits with zeros so each number of a given field is the same length. (Eg: "X1_accell(001) = 0004") This way once you find a number, you don't have to scan forward to find a non-numeric character, because you'll already know exactly where it is. You might also get better speed using VB's built-in file reading functions than the File System Objects; you should still use it to create the file, but you can open it for line input and use the Line Input function. BTW, a tip for optimization and shorter code: If chkDebug.Value = vbChecked ThenYou can use a relatively little-known trick here: assigning a boolean expression to a variable. For example, "x = (y = 2)" will assign x to the result of 'y = 2' (true if it is, false if it isn't). In this case since you're using 1 and 0 instead of true and false, you could use "debugLevel = cint((chkDebug.Value = vbChecked) * -1)" (VB defines false as -1, so you need to multiply by -1 to get 1). But better yet, vbChecked is 1, unchecked is 0, and 2 isn't user-selectable, so you could just use "debugLevel = chkDebug.Value".  |
| Add to favorites | "RSS" Feed | Next newer thread | Next older thread |
| Acmlm's Board - I2 Archive - Programming - This is never going to get fixed!! |
| | |
