[C#] Addresses. Bytes.

[gnomiux]

Hello everyone.
I recently wrote Main Analyzer(for MU) and it is my first program(in C#). So basically I didn't know anything about bytes, addresses and anything that goes along with it.

I went experimenting. Read the file, put it in the byte array. Wrote a method to turn certain search string to byte array, then added a search method on the file byte array, then some other extra stuff.. and it goes like that.

Most important thing here are the addresses. I didn't quite use it on my program directly, because I do not know how. Maybe there is some kinda memory array to manipulate with the addresses? Don't quite know what I'm talking about already, well I hope you'll understand me.

I just thought of this, that it would be interesting to know what techniques are you guys using, and what could we share to learn.

 

[Yamachi]

Are you asking about editing data at specific addresses in a file? If so, just create a Stream from the file, .Seek() to the position (address), and .Write() or .Read() whatever you need.

 

[SheenBR]

Dunno if it helps, but here is something I made for me a few months ago, its a byte sequence finder:

                var search = new byte[] { 0x2A, 0xC8, 0xBF, 0xB0, 0xFA, 0xC0, 0xBD };
                var pos = this.GetPositionAfterMatch(this.all, search);
                return this.GetValue(pos, this.all);

/// <summary>
        /// Search for a byte array inside of another one.
        /// </summary>
        /// <param name="data">the data</param>
        /// <param name="pattern">the data to look for</param>
        /// <returns>the offset of the first occurrence</returns>
        int GetPositionAfterMatch(byte[] data, byte[] pattern)
        {
            for (int i = 0; i < data.Length - pattern.Length; i++)
            {
                bool match = true;
                for (int k = 0; k < pattern.Length; k++)
                {
                    if (data[i + k] != pattern[k])
                    {
                        match = false;
                        break;
                    }
                }
                if (match)
                {
                    return i + pattern.Length;
                }
            }
            return 0;
        }

/// <summary>
        /// Return the formatted ascii value from offset
        /// </summary>
        /// <param name="pos">the position</param>
        /// <param name="data">the data to search</param>
        /// <returns>the string</returns>
        string GetValue(int pos, byte[] data)
        {
            var temp = new System.Text.StringBuilder();
            for (var i = pos; i < data.Length; i++)
            {
                if ((byte)all[i] == 0x09) continue; // we do not want tabs
                if ((byte)all[i] == 0x0D) break; // our string has finished
                temp.Append((char)all[i]);
            }
            return temp.ToString().Trim();
        }

you can edit the search method to give you the very first offset offset where your byte array begins in your file and then using the returned value to .Seek() for something just like Yamachi said.

 

[AngraMainyu]

Firstly, files do not belong in byte arrays, they belong in streams. The ReadAllBytes method may have relatively low overhead, but disk operations are expensive regardless, as is allocating the memory for large blocks of data that won't be used. In fact, I'd advise staying clear of C#'s FileStreams altogether once you're more comfortable with the language, and using the System.IO.MemoryMappedFiles APIs instead, or find a library that wraps the much faster Win32 file API -- there are plenty to choose from.

In fact, I've written a library specifically for reading executables. I haven't made any commits recently, but I have been working on the project regularly. You can find LibPE here, though I wouldn't recommend using it until I've pushed a version with import/export table support and AoB searching unless you don't mind using your own searcher, in which case you can just call GetSectionStream(".text") to obtain a stream of the code section.

You might find more help if we knew what type of addresses you were searching for.

 

[SheenBR]

well, as speed isnt my priority, I just File.ReadAllBytes() and use the function I said.

 

[AngraMainyu]

well, as speed isnt my priority, I just File.ReadAllBytes() and use the function I said.

For reading plaintext, the .NET framework already exposes ReadAllText() and ReadAllLines() which can be combined with String.IndexOf(). I could understand the need to outperform the framework implementation, but if performance isn't your goal, it seems like wasted effort.

 

[SheenBR]

I usually need to open a file in binary, and Seek for the offset, and then I convert all the bytes, starting in that offset till a specified length, to text. or I use BitConverter.GetString().

is that so bad?

 

[AngraMainyu]

I usually need to open a file in binary, and Seek for the offset, and then I convert all the bytes, starting in that offset till a specified length, to text. or I use BitConverter.GetString().

is that so bad?

I think it would depend quite a lot on the structure of the input file. For random access to C format strings, MemoryMappedViewAccessor is the only class I would recommend. Unless the input consists of short strings that are pages away from each other, it would be both more efficient and less complex than calling a disk operation for every few bytes.

 

[SheenBR]

I read on the internet that this class is good to be used when you try to acess gigabyte files. as I am only accessing nearly 40kb files, I dont think it would do any good, would it?

 

[AngraMainyu]

I read on the internet that this class is good to be used when you try to acess gigabyte files. as I am only accessing nearly 40kb files, I dont think it would do any good, would it?

Memory mapped files are without a doubt the most efficient form of file I/O available without the use of P/Invoke, regardless of the size of the file. But only if used appropriately. If, for instance, your 40KB file consisted of a table of integers, there are a few methods of reading them: for many, their first instinct would be to open a BinaryReader and make 10K calls to ReadInt32, allocating new ints as they're read in or allocating a buffer beforehand, both of which add an extra costly step to the next solution. Alternatively, you could read the data into a byte buffer and use a fixed() statement to cast it to an int array, but that buffer is now tied to another object, and every byte of that array you don't use becomes a memory leak until the object and the buffer are garbage collected. With memory mapped files, that data can be loaded in blocks of whatever size the caller desires, and can be accessed as a stream, a binary reader with generic struct support, or a byte array, and all disk operations are transparently handled by Windows. For a 40KB file, Windows might load the entire file into memory, reducing the overhead of ten thousand calls to that of one, and that is where you will see performance improvement. But again, if you're only reading small amounts of information per 4K page, you would be wasting as much memory and CPU time as reading the file into a buffer.

 

[SheenBR]

can you give me an example of how to use MMF to read a simple binary data containg strings and ints?

they are, in each file, in the same offset, I mean, their locations ( the ints and the strings ) are in the same place in each file.

 

[AngraMainyu]

can you give me an example of how to use MMF to read a simple binary data containg strings and ints?

they are, in each file, in the same offset, I mean, their locations ( the ints and the strings ) are in the same place in each file.

https://gist.github.com/1374435