Troubleshooting performance problems Apache HTTPD with PHP

I was tasked with troubleshooting some of the performance issues we were encountering for one of my projects. In a nutshell, the project is taking an board exam from a desktop-based solution to a web-based one. The test scenario was constructed using Apache JMeter, and it simulated 120 concurrent students taking the exams at the same time. While there were nothing wrong with the average response times, we were able to consistently reproduce a max response time of over 80 seconds, which is very high. It was suspicious to me because even querying the static resources took that long for the max response, which indicated to me that this might not be because of a code issue. After a couple of weeks of investigation, I looked at the logs and found this:

[Sat Sep 15 14:46:57 2012] [warn] Server ran out of threads to serve requests. Consider raising the ThreadsPerChild setting.

Right there that indicated to me that perhaps we were not allocating enough threads, and my suspicions were right....so I looked at the config under %httpd_home%/conf/extras/httpd-mpm.conf, but I could find any problems with the settings. Turns out, by default it actually disabled the inclusion of that config file in httpd.conf.

I just uncommented it out and restarted.

I then ran the test again, and voila the max time decreased dramatically to 7 seconds) which is acceptable by the client.

Disabling / turning off the bundling of js and css resources in the Grails resources plugin

I'm using the Grails resources plugin, which is great because you don't have to worry about including it in every page. However, by default it bundles all of your javascript into one big file. If you are using a whole bunch of plugins, troubleshooting that big file can be a daunting task. There is an easy to do this actually:

 modules = {  
      application {  
           defaultBundle false  
           dependsOn 'jquery'  
           resource url: "css/bootstrap/bootstrap.css"  
 ...  

Using the defaultBundle method will allow you to disable the auto-bundling feature.

Grails RabbitMQ rabbitSend not working

So I'm setting up my second app for logging purposes, and I had a little bit of an issue sending messages to RabbitMQ using Grail's rabbitmq plugin. Here was my setup:

In Config.groovy:


rabbitmq {
connectionfactory {
         ...
}

concurrentConsumers = 5

queues = {
exchange name: 'privategram.email', type: fanout, durable: true
exchange name: 'privategram.error', type: fanout, durable: true
exchange name: 'privategram.comment.created', type: fanout, durable: true
}

retryPolicy.maxAttempts = 3 //bug workaround -- see http://jira.grails.org/browse/GPRABBITMQ-34
}

As you can see, I am using an exchange with a fanout (for now).

In my BaseController.groovy:


def exception = { message ->
log.error(message)
rabbitSend('privategram.error', message)
}

In my ReceiverService:

class ErrorReceieverService {

static rabbitSubscribe = 'privategram.error'

void handleMessage(message) {

log.error "subscribed message received."

}

}

With the following set up, I was never able to send any messages. I was able to receive messages after I sent them using the RabbitMQ management console, so I knew it was something wrong with the rabbitSend method. Upon looking at the documentation, I realized that the method signature was actually wrong! Using 2 parameters means it will be sent to a queue, while using 3 parameter means it will be sent to the exchange. I personally think this api method is flawed because it is not explicit in where it is directing the messages to. But anyways this is how I fixed it:

In my BaseController.groovy:


def exception = { message ->
log.error(message)
rabbitSend('privategram.error', '', message) //notice the additional param.
}

Grails messages not working in the gsp views

So I was having this annoying issue whenever there was a validation error on one of my forms. I have configured the message key correctly and I know the feature works because I used it extensively before.

Error Field error in object 'com.privategram.User' on field 'email': rejected value [tommytcchan@gmail.com]; codes [com.privategram.User.email.unique.error.com.privategram.User.email,com.privategram.User.email.unique.error.email,com.privategram.User.email.unique.error.java.lang.String,com.privategram.User.email.unique.error,user.email.unique.error.com.privategram.User.email,user.email.unique.error.email,user.email.unique.error.java.lang.String,user.email.unique.error,com.privategram.User.email.unique.com.privategram.User.email,com.privategram.User.email.unique.email,com.privategram.User.email.unique.java.lang.String,com.privategram.User.email.unique,user.email.unique.com.privategram.User.email,user.email.unique.email,user.email.unique.java.lang.String,user.email.unique,unique.com.privategram.User.email,unique.email,unique.java.lang.String,unique]; arguments [email,class com.privategram.User,tommytcchan@gmail.com]; default message [Property [{0}] of class [{1}] with value [{2}] must be unique]

This was what I had originally.

 <g:haserrors bean="${userInstance}">  
 </g:haserrors>  
 <br />  
 <ul class="errors" role="alert">  
      <g:eacherror bean="${userInstance}" var="error">  
           <div class="alert alert-error">  
                <button class="close" data-dismiss="alert" type="button">×</button>  
                <strong>Error</strong>  
                ${error}  
           </div>  
      </g:eacherror>  
 </ul>  

...it seemed fair to me, which made me think it could be because I upgraded the version of Grails for this current project. However, upon comparing this to my other project, I realized it was an user (*cough* `me`) error. You have to wrap the error param in a g:message tag!

 <g:haserrors bean="${userInstance}">  
      <ul class="errors" role="alert">  
           <g:eacherror bean="${userInstance}" var="error">  
                <div class="alert alert-error">  
                     <button class="close" data-dismiss="alert" type="button">×</button>  
                     <strong>Error</strong>  
                     <g:message error="${error}">  
                     </g:message>  
                </div>  
           </g:eacherror>  
      </ul>  
 </g:haserrors>  

Grails: logging in user on success with Spring Security

I'm working on my next app, and the use case is that after a user signs up, he/she will be redirected to the dashboard "page". Spring security is great for adding out of the box functionality, but after combing through the documentation, it was unable to fulfill what I wanted to do. The problem is creating a user and a user role has nothing to do with the fact that the user is authenticated. I searched The Google and came across someone who had the same problem. Apparently he was directed to the helper class methods in the plugin. Turns out there is a poorly documented method, there, reauthenticate(). I didn't bother to completely read what the method did because of the name. I wanted something along the lines of authenticate(), not reauthenticate(). The documentation and the sample use case they described didn't make it apparent that it was the method to use either.

So, anyways, to make it work, my controller method looks like this:

   def save() {  
     def userInstance = new User(params)  
     if (!userInstance.save(flush: true)) {  
       render(view: "create", model: [userInstance: userInstance])  
       return  
     }  
           def userRole = Role.findByAuthority('ROLE_USER')  
           UserRole.create(userInstance, userRole, true)  
           // use this to set the context.  
           springSecurityService.reauthenticate(userInstance.email);  
     flash.message = message(code: 'default.created.message', args: [message(code: 'user.label', default: 'User'), userInstance.id])  
     redirect(controller: 'dashboard', id: userInstance.id)  
   }  

Problem with Grails Spring Security Plugin - Cannot login

I'm starting a Grails project, and I am using Spring Security to implement the login/auth feature. However, it seemed like there was no way of getting it to work, even though I have created the current roles and assigned them to the user on creation. Frustrated, I then looked at the default properties specified in the DefaultSecurityConfig.groovy file, and what I found was this:


apf.postOnly = true

Once seeing this, I had the issue fixed because I had a custom form that I created and I forgot to set the

Once I set that then everything worked!

5 easy steps to writing a Netbeans 7.x plugin

I was fed up with configuring one of the tools that we use at work. It's a data migration tool that takes data from a legacy data store and migrates it to a more modern RDBMS. Now this tool reads a whole bunch of properties from a properties file, and then uses them to configure the migration. This doesn't sound so bad on paper. The problem appears when a user has to work with multiple projects frequently. Each project requires a separate set of configuration (ie. the source/target dbs, along with a whole slew of other configurations specific for that project.) This properties file gets passed around, with people moving the keys within the file between runs. This presents a problem when someone is trying to find the differences for two given .properties files. The standard diff tool doesn't really know to sort the properties files by key first, then remove the any comments before actually performing the diff.

I was so fed up that I decided to make my own plugin in Netbeans to help with diffing two .properties files. In this post I will describe how I did it. Feel free to look at the source code locate at https://github.com/tommytcchan/properties-file-differ-netbeans

*UPDATE* - The plugin is approved and live. Check it out at: http://plugins.netbeans.org/plugin/44324/?show=true

Step 1: Create the Netbeans project.

First open Netbeans (version 7.2 as of this writing). Then click File -> New Project -> Netbeans Module -> Module. Follow the steps to create the project.

Step 2: Create the Action for someone to click to initialize the diff process.

Right click Source Packages -> New -> Action. Follow the steps to create the project. I won't go into details because this page explains it pretty well.

Step 3: Implement the logic for the diff process.

The logic for this is pretty simple. When someone clicks on the properties diff button, we pop up a file chooser dialog for the user to select the left file to compare. After they have selected a file, then we display another file chooser for the right file. After we have the two input files, we then read in the properties file, sort them by key, and then pass it into the Diff api.

I won't show the code here: if you are interested in the code, please go to project src repo:

https://github.com/tommytcchan/properties-file-differ-netbeans

One gotcha here that took me half an hour to figure out was how to include and use the various Netbeans api. To include the ones you need, you will want to right click on the project root -> Properties -> Libraries -> Module Dependencies -> Add.. and add the ones you want to use.

Step 4: Deploy the plugin so a NPM file is created.

For this part, I needed to set up the signing process before the system can build the signed NPM file.

Create a .keystore file by doing the following in a command line:


keytool -genkey -alias propertiesDiff -keypass password1

Create a LICENSE file in your project root that contains the license type of your project.

In your Module Manifest (aka manifest.mf), add in the 'OpenIDE-Module-Short-Description' key
In your Project Properties (aka proeject.properties), add in the following keys (obviously changing the values to suit your project).


keystore=~/.keystore
license.file=LICENSE
nbm.homepage=https://github.com/tommytcchan/properties-file-differ-netbeans
nbm.module.author=Tommy Chan
storepass=password1
nbm_alias=propertiesDiff

Finally, right click on the project root and click on Create NBM. In will build a *.nbm file in your `build` directory.

Step 5: Upload the plugin to the Netbeans repository.

The final step involves uploading the plugin at the Netbeans site. It's pretty self-explanatory, so I won't go into details. Go to the following url to start the process.

http://plugins.netbeans.org/plugin-publish-step1

That's it! It's pretty easy once you get the steps. I did find myself scratching my head on occasions, and searching via Google doesn't usually give me much. I hope this how-to helps somebody out there.

node-mongodb-native - Node.js - TypeError: Cannot read property 'safe' of undefined mongodb

One of the most annoying things that I found when building an app with Node.js is that you never know what's going to happen when you update a 3rd party library. Because Node is pretty new and everything is in active development, things can change dramatically. Take the noe-mongodb-native library for example. I upgraded to 1.1.x version of the library, and I started getting errors like these when I'm doing an insert:

TypeError: Cannot read property 'safe' of undefined mongodb


I looked at the implementing code and realized they changed how some of the stuff was implemented, so I locked it now to the latest version 1.1.6 and did and `npm install` and re-ran my app. Problem solved afterwards. The lesson of the day...try to lock down your apps to the specific version, or else risk debugging for half an hour trying to solve your problem!