Strophe AMD Support

A little while ago, a spent some time hacking in AMD support to Strophe.js, a client-side XMPP library. So far it hasn’t been merged, but if you are interested please comment on the pull request. It would be nice to get this added in so that we can continue to use Strophe as the world slowly moves towards more modular code using something like AMD or ES6 modules.

How to Migrate GitLab Repositories to a New Server

Just a quick post for me to refer back to in the future. I used the steps below to migrate all GitLab repositories and database from version 3.1 to 4.1.

Migrate Database

In this case, I’m using MySQL.

  1. Use mysqldump to create dump of old database, then create a new database on the new server and import this.
    1. On old: mysqldump gitlab | gzip > gitlab.sql.gz
    2. On new: gunzip < gitlab.sql.gz | mysql gitlab
  2. Run the db migrate command to make sure the schema is updated to the latest version.
    1. sudo -u gitlab -H bundle exec rake db:migrate RAILS_ENV=production

Migrate Repositories

  1. Rather than creating a new SSH key on the new server, copy over the old SSH key pair to the new server. Also copy of the authorized_keys file.
  2. Set up gitolite on the new server using the old SSH key
  3. Copy contents of /home/git/.gitolite/keydir/ from old to new server so that users don’t have to add their SSH keys again and change ownership to git:git.
  4. Copy repositories from old to new
  5. sudo chmod -R ug+rwXs,o-rwx /home/git/repositories/
  6. sudo chown -R git:git /home/git/repositories/
  7. sudo -u gitlab -H bundle exec rake gitlab:gitolite:update_keys RAILS_ENV=production
  8. sudo -u gitlab -H bundle exec rake gitlab:gitolite:update_repos RAILS_ENV=production
  9. Update projects to use namespaces. This will move any projects that are in groups to a namespace. bundle exec rake gitlab:enable_namespaces RAILS_ENV=production
    I could not get this working, easier to remove all projects from groups on the old install before migrating.
  10. Run the rake:check command and see if it complains about anything. You may need to run the create satellites command or the run the script to update the hooks.
    1. bundle exec rake gitlab:check RAILS_ENV=production
    2. sudo -u git -H /home/gitlab/lib/support/rewrite-hooks.sh

Troubleshooting

  1. If you get:
    FATAL: split conf set, gl-conf not present for ‘gitolite-admin’
    fatal: The remote end hung up unexpectedly
    Then you need to re-run the gitolite setup. This I think is because gitolite-admin is still in version 2 format.
  2. If you get an error “cannot load such file — rb-inotify”, you need to add gem “ib-notify” to the bottom of the gemfile and run bundle install again.
  3. If you get an error about invalid SSH keys, keep deleting the offending keys until the errors go away.

Unit Testing Zend_Http_Client Based Web Service Calls

This post has been sitting as a draft for ages (since before Zend Framework 2.0 ;)) but I thought I should get it out anyway!

When testing code which makes HTTP requests to web services, you do not want your tests to be reliant on the web service working. This goes against the best practices of unit tests, in which tests should be quick to run (web service might be slow), tests should ideally require no set up to run (web service calls may need API keys) and tests should be as standalone as possible (so that they can be run on say, a continuous integration server, or so you can test offline!). The point of this is so that you have no excuse not to run the tests whenever you make a change to the code. The idea of running unit tests on the client is also not to find bugs in the web service itself.

Thus, here is a basic example of a web service client class and it’s corresponding test class. By using the Zend_Http_Client_Adapter_Test adapter, we can tell Zend_Http_Client when we want it to fail, and test that the code handles failure gracefully.

Debugging ejabberd

This took me a little while to work out, and will probably be useful in the future so I thought I’d make a post about it and use my blog for the first time in a year and a half ;)

  1. Download the ejabberd source:
    git clone https://github.com/processone/ejabberd.git
  2. Configure (I want ODBC support, but you might not need it)
    cd ejabberd && ./configure --enable-odbc
  3. Compile code with debugging enabled
    make debug=true
  4. Install ejabberd; I couldn’t work out if there was a way to run ejabberd without installing it.
    sudo make install
  5. Configure ejabberd as required by editing /etc/ejabberd/ejabberd.cfg. Then run it with
    sudo ejabberdctl start
  6. Attach to the running erlang process
    sudo ejabberdctl debug
  7. Press any key to continue. Then in the shell type:
    im().

This will open up the lovely erlang debugger monitor window, as shown below.

Erlang debugger

How pretty it is!

Then go to Module -> Interpret and browse to the ejabberd source code folder from earlier, and choose the .erl file you want to debug. This only works if the beam files have been compiled with debugging enabled, using the make step from before. Once you’ve selected a file, it will appear in the list on the left and you can double click it to view it and add break points as shown below.

mod_muc_room

Adding a function breakpoint

To trigger the break point you just need to join a chat room with a client:

Break point triggered

Break point triggered

You can then step through the code by double clicking on the line with the break in it:

Step through debugging

Step through debugging

Happy debugging!

ejabberd mod_archive with MySQL on Ubuntu

  1. Install the MySQL ODBC driver,

    apt-get install libmyodbc

  2. Create an ODBC configuration for ejabberd at /etc/odbc.ini,
    [ODBC Data Sources]
    odbcname     = MyODBC 3.51 Driver DSN
    
    [ejabberd]
    Driver       = /usr/lib/odbc/libmyodbc.so
    Description  = MyODBC 3.51 Driver DSN
    SERVER       = localhost
    PORT         =
    USER         = ejabberd
    Password     = ejabberd
    Database     = ejabberd
    OPTION       = 3
    SOCKET       =
  3. Download mod_archive from Subversion.
  4. svn co https://svn.process-one.net/ejabberd-modules
    cd ejabberd-modules/mod_archive/trunk
  5. You may then have to edit Emakefile before compiling. This is because by default it will point to the include files from ejabberd trunk, which will probably be for a newer version of ejabberd than you have installed, unless you installed ejabberd from trunk. in Emakefile replace all instances of:
  6. trunk

    With

    branches/ejabberd-2.0.x

    And then run

    ./build.sh
  7. This will generate some *.beam files in the ebin directory. Copy these to the ejabberd ebin directory which should contain all the other ejabberd .beam files. The directory for me is /usr/lib/ejabberd/ebin/.
  8. Create the MySQL database schema using the SQL scripts from mod_archive. They are in ejabberd-modules/mod_archive/trunk/src/*.sql. The script for MySQL is mod_archive_odbc_mysql.sql.
  9. Add in the configuration options to enable mod_archive in ejabberd.cfg, under modules:
  10. {mod_archive_odbc, []},
  11. Create ODBC connection:
  12. {odbc_server, "DSN=ejabberd;UID=ejabberd;PWD=ejabberd"}.

    Note that the authentication won’t use this by default because in the configuration file there is an option:

    {auth_method, internal}.

    Which means it will use the internal mnesia database for auth.

  13. Finally, to view the archives in a web browser, add to the listen section of the ejabberd.cfg file:
    {request_handlers, [{["archive"],mod_archive_webview}]}

    So that the final section for port 5280 looks something like

      {5280, ejabberd_http, [
    			 http_poll,
    			 web_admin,
                             {request_handlers, [{["archive"],mod_archive_webview}]}
    			]}
    
     ]}.
  14. The archives should then be visible at http://host:5280/archive/

More info

Vital ejabberd Notes

Making Changes Stick

Put

override_global.
override_local.
override_acls.

At the top of the ejabberd.cfg file to ensure that changes in the configuration file take affect. Otherwise, you may experience strange results where changes do not seem to take affect after restarting the server. This is because the configuration file is loaded into the internal ejabberd database. If you do not override certain settings in the configuration file, the old values from the database will be used. Spent much time trying to work out why my changes to the configuration file weren’t being noticed.

Controlling ejabberd

Erlang, the language ejabberd is written in, uses nodes. These are controlled with ejabberdctl. ejabberd itself can be controlled with the init script, /etc/init.d/ejabberd. If you get:

Node ‘ejabberd@host’ is started. Status: started
ejabberd is not running

That means the node is running, but inside that ejabberd didn’t start correctly, but didn’t crash. Usually this is a misconfiguration, and the error message will be in a log file, either /var/log/ejabberd/ejabberd.log or /var/log/ejabberd/sasl.log. When making changes to the configuration file, usually you just use ejabberdctl restart, and don’t use the init script.