Category: development

  • Encrypting application configuration files – Java AES

    There other day I needed to encrypt certain properties in our XML configuration files. But there were few challanges that needed to be addressed

    • Use existing Java installation
    • !!! How will Master Encryption Key be derives and stored !!!

    First restriction is very important as it limits us to what algorithmic and key sizes can be used, adding Java Cryptography Extension was not a option.

    This limited us to using AES with key size 128, using 256 would give us an exception.

    java.security.InvalidKeyException:Illegal key size or default parameters
    

    Second important question raised was “How will the master password be derived and possibly stored ?”, there are a number of approaches to this and each have its advantages, but the constraint here is that there should be no user interaction involved.

    Few of the approaches considered

    Passing password as a environment variable
    This requires users to modify the startup parameters and the password is plain text
    Prompting user to provider password when they start the application
    This is good method(KeePass use it) but it requires user interaction, which was not acceptable in my case
    Generating unique machine key
    This method is not the best but it does work well, as long as the attacker don’t know how the key was generated.

    Approach chosen by me was to use the machine generated key/id, I have chooses the MAC address as the machine key, I know it can be easily spoofed but if the attacker is that dedicated then I am screwed anyway.

    Machine key generation algorithm is is straight forward. First we will try to obtain the MAC Address from the network interface, if that fails we we try to use the machine name obtained from the RuntimeMXBean, this is very depended on what JVM we are using as there is no guarantee what getName() method will return, if this fails we will fall back to using
    System.getenv("COMPUTERNAME"), if all this fails we simply throw an exception.

    Encrypted Message Format

    CRYPT:Ss3iHK6kHLswAh7AyoHdo1dbbTJt0UpLxVNRZ9W+bks=

    Encrypted messages can be broken down into three parts

    Identifier : CRYPT used to indicate that this property have been encrypted
    Initial Vector + Data : Ss3iHK6kHLswAh7AyoHdo1dbbTJt0UpLxVNRZ9W+bks=

    Because we know the IV size(128 bits) we can prepend that to the begging of our encrypted message, so when we are decrypting the message we know that first 128bits will be the IV.

    Example Usage

      AESEncryptDecryptUtil util = new AESEncryptDecryptUtil();
            String encrypedMessage = util
                .encrypt("Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin scelerisque sodales augue, ac tristique nibh euismod nec. ");
            boolean status = util.isEncrypted(encrypedMessage);
    
            System.out.println("Encrypted : " + encrypedMessage);
            System.out.println("Message encrypted : " + status);
            System.out.println("Decrypted : " + util.decrypt(encrypedMessage));
    

    Encrypting Properties

       public String getUserName()
        {
            try
            {
                AESEncryptDecryptUtil util = new AESEncryptDecryptUtil();
                if (util.isEncrypted(userName))
                {
                    return util.decrypt(userName);
                }
            }
            catch (EncryptDecryptException e)
            {
                e.printStackTrace();
            }
            return userName;
        }
    

    Output

    Encrypted : CRYPT:keGhiojDBpgsUkdepcT6aFvzR0Jpsmi4no3TwP3OzFbAmjvvBxkSQHMyQ8uM81PTCTEWebe3HrE5
    o/+jLgdmQB+CDxBMUvUrFRvFFAGsrCcS+cWgYTi+T5/LTUFonXBOc5r++GQbV2htxy9YislL2JPT
    vMQAGrgMjOFTDBZvuJdeUIC6uKWjtndJidxigxHB
    Message encrypted : true
    Decrypted : Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin scelerisque sodales augue, ac tristique nibh euismod nec.
    
    

    Code Listing

    
    import java.io.UnsupportedEncodingException;
    import java.lang.management.ManagementFactory;
    import java.lang.management.RuntimeMXBean;
    import java.net.InetAddress;
    import java.net.NetworkInterface;
    import java.net.SocketException;
    import java.net.UnknownHostException;
    import java.security.AlgorithmParameters;
    import java.security.InvalidKeyException;
    import java.security.spec.InvalidParameterSpecException;
    import java.security.spec.KeySpec;
    
    import javax.crypto.BadPaddingException;
    import javax.crypto.Cipher;
    import javax.crypto.IllegalBlockSizeException;
    import javax.crypto.SecretKey;
    import javax.crypto.SecretKeyFactory;
    import javax.crypto.spec.IvParameterSpec;
    import javax.crypto.spec.PBEKeySpec;
    import javax.crypto.spec.SecretKeySpec;
    
    import sun.misc.BASE64Decoder;
    import sun.misc.BASE64Encoder;
    
    /**
     * AES Encryption/Decryption Utility that uses MAC address as the Master
     * Encryption Key Initial Vector need to be stored together with the encrypted
     * string so that we can use it to decrypt the message. This is the format that
     * will be produced from the encryption, both encrypted message and initial
     * vector are base 64 encoded, in UTF-8 char set
     * CRYPT:initialVector+encryptedMessage
     * 
     * @author gbugaj
     */
    public class AESEncryptDecryptUtil
    {
    
        private static final int ITERATIONS = 65536;
    
        private static final String STRING_ENCODING = "UTF-8";
    
        private static final String CRYPT_PREFIX = "CRYPT";
    
        /**
         * If we user Key size of 256 we will get java.security.InvalidKeyException:
         * Illegal key size or default parameters , Unless we configure Java
         * Cryptography Extension 128
         */
        private static final int KEY_SIZE = 128;
    
        private static final byte[] SALT = { (byte) 0x0, (byte) 0x0, (byte) 0x0, (byte) 0x0, (byte) 0x0, (byte) 0x0, (byte) 0x0, (byte) 0x0 };
    
        private SecretKeySpec secret;
    
        private Cipher cipher;
    
        private BASE64Encoder base64Encoder;
    
        private BASE64Decoder base64Decoder;
    
        private AESEncryptDecryptUtil() throws EncryptDecryptException
        {
            try
            {
                /* Derive the key, given password and salt. */
                SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
                KeySpec spec;
    
                spec = new PBEKeySpec(getHardwareKey(), SALT, ITERATIONS, KEY_SIZE);
                SecretKey tmp = factory.generateSecret(spec);
                secret = new SecretKeySpec(tmp.getEncoded(), "AES");
    
                // CBC = Cipher Block chaining
                // PKCS5Padding Indicates that the keys are padded
                cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
    
                // For production use commons base64 encoder
                base64Decoder = new BASE64Decoder();
                base64Encoder = new BASE64Encoder();
            }
            catch (Exception e)
            {
                throw new EncryptDecryptException("Unable to initialize", e);
            }
    
        }
    
        /**
         * Get hardware key to be used for encryption, we rely on MAC address as the
         * key, with fallback to JVM name then windows machine name
         * 
         * @return key as char[]
         * @throws EncryptDecryptException
         */
        private char[] getHardwareKey() throws EncryptDecryptException
        {
    
            // Exceptions from this are ignored as we move to next available method
            try
            {
                InetAddress address = InetAddress.getLocalHost();
                NetworkInterface nic = NetworkInterface.getByInetAddress(address);
                if (nic != null)
                {
                    byte[] mac = nic.getHardwareAddress();
                    if (mac != null && mac.length > 0)
                    {
                        return new String(mac, STRING_ENCODING).toCharArray();
                    }
                }
            }
            catch (UnknownHostException e)
            {
                e.printStackTrace();
            }
            catch (SocketException e)
            {
                e.printStackTrace();
            }
            catch (UnsupportedEncodingException e)
            {
                e.printStackTrace();
            }
    
            // Could not obtain MAC Address so we are falling back on the computer
            // name then on the JVM Name
            RuntimeMXBean rmx = ManagementFactory.getRuntimeMXBean();
            String jvmName = rmx.getName();
            String[] parts = jvmName.split("@");
            if (parts.length > 0)
            {
                String name = parts[1];
                if (name != null && !name.isEmpty())
                {
                    return name.toCharArray();
                }
            }
    
            String name = System.getenv("COMPUTERNAME");
            if (name != null && !name.isEmpty())
            {
                return name.toCharArray();
            }
            throw new EncryptDecryptException("Unable to obtain Secure Key");
        }
    
        /**
         * Encrypt given input string
         * 
         * @param input
         * @return
         * @throws EncryptDecryptException
         */
        public String encrypt(String input) throws EncryptDecryptException
        {
            try
            {
                byte[] inputBytes = input.getBytes(STRING_ENCODING);
                cipher.init(Cipher.ENCRYPT_MODE, secret);
                AlgorithmParameters params = cipher.getParameters();
                byte[] iv = params.getParameterSpec(IvParameterSpec.class).getIV();
                byte[] ciphertext = cipher.doFinal(inputBytes);
                byte[] out = new byte[iv.length + ciphertext.length];
                System.arraycopy(iv, 0, out, 0, iv.length);
                System.arraycopy(ciphertext, 0, out, iv.length, ciphertext.length);
                return CRYPT_PREFIX + ":" + base64Encoder.encode(out);
            }
    
            catch (IllegalBlockSizeException e)
            {
                throw new EncryptDecryptException("Unable to encrypt", e);
            }
            catch (BadPaddingException e)
            {
                throw new EncryptDecryptException("Unable to encrypt", e);
            }
            catch (InvalidKeyException e)
            {
                throw new EncryptDecryptException("Unable to encrypt", e);
            }
            catch (InvalidParameterSpecException e)
            {
                throw new EncryptDecryptException("Unable to encrypt", e);
            }
            catch (UnsupportedEncodingException e)
            {
                throw new EncryptDecryptException("Unable to encrypt", e);
            }
        }
    
        /**
         * Decrypt input string
         * 
         * @param input
         * @return decrypted string
         * @throws EncryptDecryptException
         */
        @SuppressWarnings("restriction")
        public String decrypt(String input) throws EncryptDecryptException
    
        {
            if (!input.startsWith(CRYPT_PREFIX))
            {
                throw new EncryptDecryptException("Unable to decrypt, input string does not start with 'CRYPT'");
            }
    
            try
            {
                byte[] data = base64Decoder.decodeBuffer(input.substring(6, input.length()));
                int keylen = KEY_SIZE / 8;
                byte[] iv = new byte[keylen];
                System.arraycopy(data, 0, iv, 0, keylen);
                cipher.init(Cipher.DECRYPT_MODE, secret, new IvParameterSpec(iv));
                return new String(cipher.doFinal(data, keylen, data.length - keylen), STRING_ENCODING);
            }
            catch (Exception e)
            {
                throw new EncryptDecryptException("Unable to decrypt ", e);
            }
        }
    
        /**
         * Helper Exception to wrap arround Encryption/Decryption exceptions
         */
        @SuppressWarnings("serial")
        private static class EncryptDecryptException extends Exception
        {
            public EncryptDecryptException()
            {
                super();
            }
    
            public EncryptDecryptException(String msg)
            {
                super(msg);
            }
    
            public EncryptDecryptException(String msg, Throwable t)
            {
                super(msg, t);
            }
    
            public EncryptDecryptException(Throwable t)
            {
                super(t);
            }
        }
    
        /**
         * Check if given string is already encrypted
         * 
         * @param input
         * @return
         */
        private boolean isEncrypted(String input)
        {
            if (input == null || input.isEmpty())
            {
                return false;
            }
            return input.startsWith(CRYPT_PREFIX);
        }
    
        public static void main(String[] args) throws EncryptDecryptException
        {
            AESEncryptDecryptUtil util = new AESEncryptDecryptUtil();
            String encrypedMessage = util
                .encrypt("TEST");
            boolean status = util.isEncrypted(encrypedMessage);
    
            System.out.println("Encrypted : " + encrypedMessage);
            System.out.println("Message encrypted : " + status);
            System.out.println("Decrypted : " + util.decrypt(encrypedMessage));
        }
    
    }
    
  • Population Count using popcnt instruction (counting number of on bits)

    Using popcnt instruction found on newer processors to get population count which is number of set bits.

    Reference : SSE4

    Here are couple example inputs with expected output.

     0xA =  1010 -- > 2
     0x3F2 =  1111110010 -- > 7	
    
    #include 
    #include 
    
    using namespace std;
    
    int main() {
    	// 0xA =  1010 -- > 2
    	// 0x3F2 =  1111110010 -- > 7
    	
    	unsigned long code = 0x3F2UL;
    	int bitcount = 0;
    
    	__asm__( "movl %1, %%eax; "  //code into eax
    			"popcnt  %%eax,  %%eax;"//  call popcnt instruction
    			"movl %%eax, %0;"// ebx into bitcount
    			:"=r"(bitcount)// output
    			:"r"(code)// input
    			:"%eax","%ebx","%ecx","%edx"// clobbered register
    	);
    
    	printf("%d", bitcount);
    	return 0;
    }
    

    Update
    To answer Phils question why we have "%eax","%ebx","%ecx","%edx" as our clobbered register list, this is my understanding and it could be completely wrong.

    Our instruction movl %%eax, %0; used GCC syntax for accessing c++ variable bitcount. So ‘%0’ refers to bitcount which will be copied into ebx register, that is why I have ebx in the list, as for the ecxand edx they have been added purely because they are general purpose registers and I did not want to override some previously stored value and my limited understanding on how inline assembly works.

  • Leptonica Tutorial – Cropping

    Common use is to crop to certain area of an image, this can be quite easily accomplished with following code.

    BOX* box = boxCreate(x, y, w, h);
    PIX* dpix= pixClipRectangle(spix, box, NULL);
    
  • Using ‘cpuid’ instructions to get cache line size

    This is program demonstrates how to use ‘cpuid’ assembly instruction on x86 to retrieve cache line size. This is done in c++ and it uses the inline assembly to execute the instructions, compiled with gcc on win32.

    Really we are only interested in Cache line information which can be found using 0x80000006 code and its lower 8 bits returned from ECX register.

    #include 
    #include 
    #include 
    
    int MASK_LSB_8 = 0xFF;
    
    void int2bin(int val) {
    	char buffer[33];
    	itoa(val, buffer, 2);
    	printf("binary: %s\n", buffer);
    }
    
    // CPUID instruction takes no parameters as CPUID implicitly uses the EAX register.
    // The EAX register should be loaded with a value specifying what information to return
    void cpuinfo(int code, int *eax, int *ebx, int *ecx, int *edx) {
    	__asm__ volatile(
    			"cpuid;" //  call cpuid instruction
    			:"=a"(*eax),"=b"(*ebx),"=c"(*ecx), "=d"(*edx)// output equal to "movl  %%eax %1"
    			:"a"(code)// input equal to "movl %1, %%eax"
    			//:"%eax","%ebx","%ecx","%edx"// clobbered register
    	);
    }
    
    int main() {
    	int eax = 0, ebx = 0, ecx = 0, edx = 0;
    	//CPUID(0): Basic information
    	cpuinfo(0x0, &eax, &ebx, &ecx, &edx);
    
    	// check vendor [GenuineIntel, GenuntelineI] = ebx+ecx+edx
    	char vendor[13] = { 0 };
    	int identity_reg[3] = { ebx, ecx, edx };
    	for (unsigned int i = 0; i < 3; i++) {
    		for (unsigned int j = 0; j < 4; j++) {
    			vendor[i * 4 + j] = identity_reg[i] >> (j * 8) & MASK_LSB_8;
    		}
    	}
    	printf("\nVendor name = %s\n", vendor);
    
    	//CPUID(0x80000000): Extended CPU information
    	cpuinfo(0x80000000, &eax, &ebx, &ecx, &edx);
    	int intelExtended = eax & MASK_LSB_8;
    	printf("\nMax feature id = %d\n", intelExtended);
    	if (intelExtended < 6) {
    		printf("\nCache Extended Feature not supported");
    		return 0;
    	}
    	// CPUID(0x800000006): Cache line information
    	cpuinfo(0x80000006, &eax, &ebx, &ecx, &edx);
    	// We are only interested in lower 8 bits
    	printf("\nL2 Cache Size = %d\n", (ecx & MASK_LSB_8));
    }
    

    Here is the output from my machine.

    Vendor name = GenuntelineI
    Max feature id = 8
    L2 Cache Size = 64
    

    The following result can be verified by using cpu-z program.

      	L1 Data cache	4 x 32 KBytes, 8-way set associative, 64-byte line size
        	L1 Instruction cache	4 x 32 KBytes, 8-way set associative, 64-byte line size
        	L2 cache	4 x 256 KBytes, 8-way set associative, 64-byte line size
        	L3 cache	6 MBytes, 12-way set associative, 64-byte line size
    

    References

  • Leptonica Tutorial – Environment setup

    Leptonica Development with Eclipse

    This is the first one of few tutorials that I am writring on using Leptonica image processing library

    Before we start let’s give credit where the credit is due, and this is goes out to Jay W.(need link) on which this instructions are largely based on.

    Environment

    • Eclipse
    • MinGW
    • Leptonica

    I assume that you have installed and configured Eclipse CDT (http://www.eclipse.org/cdt/) plugging for C/C++ development, as well as MinGW and can run simple c++ hello world using make file.

    Setup

    1. Lets start by downloading latest leptonica library from Leptonica download site
    and extract the library into your lib directory I use E:/source/c-libraries/leptonica-1.68

    2. This is probably the most complicated part of the setup, configuring the Eclipse environment and compiling Leptonica hello world project.

    1. Lets start by ‘Creating new C++ Make Project’ and naming it ‘HelloWorldLeptonica’ in Eclipse, before continuing lets make sure we can compile and run the project.
    2. Now we will edit Properties for our newly created project, navigate to Properties->C/C++ Build -> Environment
    and add new property LEPT_HOME

    LEPT_HOME E:/source/c-libraries/leptonica-1.67
    

    Include/Library Configuration

    Navigate to Properties->C/C++ General -> Paths and Symbols and select Include Tab -> GNU-C++
    and add new Include directory ‘E:/source/c-libraries/leptonica-1.67/include’, this will allow us to browse includes from within Eclipse.

    Now lets copy leptonlib.dll from ‘E:/source/c-libraries/leptonica-1.67/lib’ into our project directory.
    this way we can run the project directly from Eclipse.

    Makefile Configuration

    I will highlight main sections that need to be configured, this is where we use our previously defined variable ‘LEPT_HOME’ come into play.

    • Include Files – INCS
    • Libraries – LIBS
    • Compiler Flags – CFLAGS

    We need to make sure to include header files and leptonica library.

    INCS		=	-I$(LEPT_HOME)/include		
    LIBS		=	-L$(LEPT_HOME)/lib		\
    			-lleptonlib			
    

    Capital flag -L indicates to includes $(LEPT_HOME)/lib directory in the search path, while lowercase -l indicates what library to load, if we wanted to include debug version we would use -lleptonlibd

    Here is a make file used by the demo project

    
    CFLAGS =	-O2 -g -Wall -fmessage-length=0
    CPP         =   g++
    OUTPUT_DIR	=	.
    OBJS		=	HelloWorldLeptonica.o
    INCS		=	-I$(LEPT_HOME)/include		
    LIBS		=	-L$(LEPT_HOME)/lib		\
    			-lleptonlib		
    PROJECT =	HelloWorldLeptonica.exe
    OUTPUT		=	$(PROJECT)
    
    all: $(PROJECT)
     
    $(PROJECT):	$(OBJS)
    	@echo Linking $(OUTPUT)
    	@$(CPP) $(LFLAGS) -o $(OUTPUT_DIR)/$(OUTPUT) $(OBJS) $(LIBS)
    	@echo Completed building $(OUTPUT)
     
    clean:
    	@echo Cleaning $(PROJECT)
    	@rm -rf $(OBJS) $(OUTPUT_DIR)/$(PROJECT) $(OUTPUT_DIR)/$(PROJECT).so $(PROJECT).so $(PROJECT) $(PROJECT).exe *.d
     
    -include $(OBJS:.o=.d)
     
    %.o:	%.cpp
    	@echo Compiling $<
    	@$(CPP) $(CFLAGS) -c $(INCS) -o $*.o $*.cpp
    	@$(CPP) -MM $(CFLAGS) $(INCS) $*.cpp > $*.d
     
    %.o:	%.c
    	@echo Compiling $<
    	@$(CC) $(CFLAGS) -c $(INCS) -o $@ $<
    	@$(CC) -MM $(CFLAGS) $(INCS) $*.c > $*.d
    
    	
    

    Example

    As always we want to start with something simple so here is it, we get current directory and read in file a ‘test.png’ file, if all went well
    we display image dimensions.

    /**
     * Leptonica Hello World
     */
    #include 
    #include 
    #include 
    
    #include 
    #include 
    #include 
    
    #if !defined(LEPTONICA_ALLHEADERS_H)
    #   include 
    #endif
    
    int main(void) {
    	printf("\nTesting Leptonica Setup \n");
    
    	// Read test.png from current working directory
    	char* filename = NULL;
    	filename = getcwd(filename, MAXPATHLEN);
    	strcat(filename, "\\test.png");
    
    	PIX* pix = pixRead(filename);
    	if (!pix) {
    		printf("Error opening file");
    		return 1;
    	}
    	printf("Image width : %d", pix->w);
    	pixDestroy(&pix);
    	return 0;
    }
    

    If we want to build the project from outside of Eclipse you will have to define the the make file environmental variable in make file directly

     LEPT_HOME=E:/source/c-libraries/leptonica-1.67
    

    g++ and Eclipse debugging hint

    In am not an expert in c++ in any way so this can be totally wrong, but in order to get the debugger to work in eclipse
    the Compiler Flags (CFLAGS) need to be using -g -O0 instead of the default one -O2 -g.
    If we don’t do this then Eclipse complains that it can’t find the source when it hits a breakpoint.

    Download HelloWorldLeptonica Eclipse Project from here HelloWorldLeptonica

    At this point you should be ready to compile and run/debug leptonica project from within Eclipse IDE.

    As always comments and corrections are always welcome.