Kevin Glowacz

Rails Debugging with VS Code and puma-dev

2024-03-10T00:00:00+00:00

Rails debugging has gotten really good lately; a new Rails project automatically includes the ruby debug gem, and it is enabled by default in development. The Rails instructions for it work great if you start your server with rails server and want to use the command line debugging interface. However, I prefer to run all my apps with puma-dev to give myself a nice local test domain name, and I prefer to use vscode for debugging.

Luckily, with just a little bit of code, we can configure the debug gem to automatically open a named debug socket on app launch that we can attach to via vscode and the vscode-rdbg extension.

Add the following lines to your config/environments/development.rb file:

# config/environments/development.rb
Rails.application.server do
  DEBUGGER__.open(nonstop: true, session_name: "my-app-server")
end
Rails.application.console do
  DEBUGGER__.open(nonstop: true, session_name: "my-app-console")
end

I want to call out a little known Rails feature that I’m taking advantage of here. The .console and .server code blocks are handy built in Rails methods that facilitate running code on boot specifically when in the context of a console session or server session. By using them, I can easily specify different debug session names for each context.

DEBUGGER__.open is the method to manually open a debug session. By specifying nonstop: true we are telling the debugger to not stop the process upon launch which is important so that the app can run like normal if there are no attached debug sessions or breakpoints. The session_name parameter allows a custom string to be appended to the unix socket and is used to identify the session in vscode when attaching to the debugger while multiple sessions are open.

To attach to the debugger in vscode, you will need to install the vscode-rdbg extension and add a launch configuration to your .vscode/launch.json file. Here is an example of what that might look like:

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "rdbg",
      "name": "Attach with rdbg",
      "request": "attach",
    }
  ]
}

Now when you start the debug session in vscode, if there is more than one active debug session you will be prompted to select which one you want to attach to. The custom session names make it easy to find the right one. You can then set breakpoints and inspect variables as you would expect.

Hacking the Messages Database

2023-12-20T00:00:00+00:00

With the release of iOS 17.2 and macOS 14.2, there is now the ability to see a count of the number of chat messages stored in iCloud. A bunch of people were comparing counts and what not both on social media and in work chats, so I decided to take a look at my count too. I noticed that for some reason my number was an order of magnitude higher than everyone else’s. While many people had hundreds of thousands of messages, I had 2.9 Million. I am not a popular individual, so I knew something had to be wrong. I decided to investigate by hacking the messages database.

After some internet searching, I found that the messages database on the Mac is stored at ~/Library/Messages/chat.db. It’s a pretty simple SQLite database that can be accessed via a number of different db utilities including the sqlite3 command line tool. The trick is that whatever app is accessing the database needs to be granted full disk access in the security settings. My preferred tool for looking at databases is TablePlus. After a quick group by query into the message table, it became clear that the vast majority of these messages are from SMS.

select service, count(1) from message group by 1;

(Wow, look at those old Jabber messages, I had no idea they were still in there)

Honing in on the SMS messages, I found that 2.8 Million of these messages contained no text content, and had a value of 39 in the error column. Also they all seemed to have a date value of 489239966000000000. It appears that this value is the number of nanoseconds since 2001-01-01, so it correspond to Sunday, July 3, 2016. I have no idea what’s going on with the date. Maybe it is some sort of default value, or my maybe account had some sort of bug at that time and flooded my database with these erred messages.

My next thought is, of course, how do I clean up just these specific messages? I want them removed everywhere, not just my Mac, but iCloud too. These messages are not associated with any particular chat, so I can’t just use the Messages UI, instead I try to see if I can just delete them from the database. I try the following query:

DELETE FROM "message" WHERE "ROWID" = <id goes here>';

When I tried this, I got the error: no such function: after_delete_message_plugin. It appears that this database has a number of triggers defined that do various things when items are deleted from the database. In this case, the message delete trigger is trying to call a function, but apparently this function isn’t defined in the database. I learned that SQLite has this feature called Application-Defined SQL Functions. This feature allows apps to define custom SQL functions that call back into the application code. I assume that Messages must define the after_delete_message_plugin function somewhere within the app code, so I don’t have access to it when talking directly to the db outside of the app context. I searched for a bit to see if there was some way to override this function, but I couldn’t find anything quickly. I decided to try a different approach. What if instead, I associate each of these messages with a chat, then delete the chat in the app!

In order to do this, I found a chat that I will willing to delete: one of those SMS shortcode chats that sent me a one time access code or something else automated. I then crafted a query that would associate all these bad SMS messages with that chat through the chat_message_join table.

INSERT INTO chat_message_join (chat_id, message_id, message_date)
SELECT
	1018,
	ROWID,
	date
FROM
	message
	LEFT OUTER JOIN chat_message_join ON chat_message_join.message_id = ROWID
	LEFT OUTER JOIN chat_recoverable_message_join ON chat_recoverable_message_join.message_id = message.ROWID
WHERE
	date = 489239966000000000
	AND account_guid IS NULL
	AND service = 'SMS'
	AND account IS NULL
	AND text IS NULL
	AND error = 39
	AND handle_id = 0
	AND attributedBody IS NULL
	AND chat_message_join.chat_id IS NULL
	AND chat_recoverable_message_join.chat_id IS NULL
	AND balloon_bundle_id IS NULL
ORDER BY
	ROWID
LIMIT 500000

This query selects all the bad SMS messages (I added some extra conditions just to be extra sure I wasn’t deleting anything that could be important), and inserts them into the chat_message_join table with the chat ID of the chat I want to delete (in this case 1018). By joining to the chat_message_join and chat_recoverable_message_join tables and selecting on NULL values, I also ensure that I don’t insert any new message joins for messages that are already joined to a chat. I also limit the query to 500,000 messages at a time, because I have no idea what Messages will do when trying to delete that extreme number of messages all at once.

After relaunching Messages, the chat in question now shows all these blank messages in their erred state within a chat window.

Now for the moment of truth, I delete the chat.

Message got stuck in this exact state showing the trash can in a red background for many hours while the IMDPersistenceAgent process pegged a CPU core. Whatever the app is doing, it seems like it must be doing it synchronously, for each message, maybe it has something to do with that after_delete_message_plugin function.

Eventually, the process finished and the chat was deleted. It showed in the Deleted Chat view with all of the messages still linked, and I could see all the associations in the chat_recoverable_message_join table.

(I did a 1,000 and 10,000 test batch before doing the full 500,000)

Deleted messages say they take 30 days before the are no longer recoverable on device, and then 10 more days after that before they are deleted from iCloud. To speed this process up, I force deleted the chat out of Recently Deleted. This seemed to remove all the chat_recoverable_message_join associations as well as the original message records from the local database. The operation completed very quickly (relative to the process of deleting the chat in the first place).

After this I forced iCloud syncs on all my devices, and this put my phone into a very hot state. The battery drained very quickly and I even managed to catch this interesting error message:

iCloud still reports the same number of message in the cloud and my other computer has not had these messages deleted from its local database, so I’m not entirely sure yet if this actually did anything useful, but the fact that my phone overheated leads me to believe that it at least did something. I guess I’ll find out in 10 days if the iCloud count goes down or not.

Just for fun, I turned off the iCloud sync on the original Mac and then turned it back on. The sync process re-download all those deleted messages, so I’m thinking that might not be a good sign that this worked the way I wanted. I’ll make an update here after 10 days or if I learn anything new.

Do you have any contacts on the Message team or have any insight here? If so please reach out to me on Mastodon. I have also filed this as FB13485744

Update: 2023-01-09

It has been more than 10 days and the number of messages in iCloud has not gone down, so apparently my process for deleting these messages did not work.

Checking on the status of the TouchPad, polling for changes on a web page

2011-08-25T16:52:00+00:00

With all the hype surrounding the price drop of the HP TouchPad, I have found myself spending a lot of time trying to find a store with one in stock. At one point I saw that HP's own store listed the touchpad as "coming soon" after most stores had run out of stock. Instead of refreshing the page on my own I wrote a script to do it for me.

This script loads the product page every 30 seconds and checks the button area to see if it still says coming soon or if it has the out of stock image. When it no longer finds either of those things, it has my computer say "alert" over and over again (in an australian accent, because who doesn't change their speech voice to australian) and automatically loads the product page in Chrome Canary for me.

I'm sure there are many others like it, but this one is mine. Check it out. Use it, fork it, modify it for your own needs. I license this under WTFPL.

A simple irc bot in ruby

2009-03-07T19:55:00+00:00

Recently it occurred to my colleagues and me that an IRC bot could benefit our chat room discussions. I looked around a bit and I couldn't find any simple ruby IRC bots to use. However, in my searching, I did find shout-bot. Shout-bot is a simple IRC "shouter" in that it connects, reports a message, and then disconnects. Using this as a starting point, I was able to create a simple bot that stays connected to a room and responds to messages.

This bot isn't very complex and is far from perfect, but it is a good starting point if you want an irc bot that responds to just a few things. You can also find this at its github repository.

#!/usr/bin/env ruby

require 'socket'

class SimpleIrcBot

  def initialize(server, port, channel)
    @channel = channel
    @socket = TCPSocket.open(server, port)
    say "NICK IrcBot"
    say "USER ircbot 0 * IrcBot"
    say "JOIN ##{@channel}"
    say_to_chan "#{1.chr}ACTION is here to help#{1.chr}"
  end

  def say(msg)
    puts msg
    @socket.puts msg
  end

  def say_to_chan(msg)
    say "PRIVMSG ##{@channel} :#{msg}"
  end

  def run
    until @socket.eof? do
      msg = @socket.gets
      puts msg

      if msg.match(/^PING :(.*)$/)
        say "PONG #{$~[1]}"
        next
      end

      if msg.match(/PRIVMSG ##{@channel} :(.*)$/)
        content = $~[1]

        #put matchers here
        if content.match()
          say_to_chan('your response')
        end
      end
    end
  end

  def quit
    say "PART ##{@channel} :Daisy, Daisy, give me your answer do"
    say 'QUIT'
  end
end

bot = SimpleIrcBot.new("irc.freenode.net", 6667, 'ChannelName')

trap("INT"){ bot.quit }

bot.run

Coda Plug-in: Remove Trailing Whitespace

2008-12-10T05:30:00+00:00

Edit: Erik Hinterbichler has written a much better native plugin called White Out. I highly recommend using that one instead of this one.

I've been using Coda to do all of my programming for quite some time now. One thing I wish it had, though, is the ability to remove trailing white space from lines when the file is saved.

With Coda 1.6 they added the ability to create plug-ins, and with the release of 1.6.1 a few days ago they have added the ability to manipulate the whole document with plug-ins. After hearing this I decided to write my own to take care of that pesky trailing white space issue. Coda doesn't have the ability to have plug-ins run upon file save, but it's the best I could do with what they provide and what I know how to do.

You can download the plug-in here:

Remove Trailing Whitespace v1.0

Here is the code I used to write it. The $$IP$$ stuff is so that the insertion point remains where it was before the code is run. If I leave it out, the insertion point goes to the end of the file. You can also visit its github repository.

#!/usr/bin/ruby

ip_line = ENV['CODA_LINE_NUMBER'].to_i
ip_index = ENV['CODA_LINE_INDEX'].to_i

$stdin.each_line do |line|
  line.rstrip!

  if $stdin.lineno == ip_line
    ip_index = line.length if (line.length) < ip_index
    line = line.insert(ip_index, "$$IP$$")
  end

  print line
  print ENV['CODA_LINE_ENDING']
end

Dynamic Delicious Share Link for Blogger

2008-11-20T04:40:00+00:00

I was working on adding share links to my blog posts and I discovered that both Digg and reddit have pretty nice dynamic share buttons with instruction on how to use them. These buttons show the number of diggs or up-votes obtained by the article if they have already been submitted.

While delicious also has some nice buttons, the one that shows the number of times an article has been bookmarked does not allow for a url to be specified. It only works for the location of the page it is on. This means I can't put it at the bottom of each post and have it work when multiple posts are displayed.

After looking around, I found a site that uses the Delicious API to grab the total post count. The code he uses though is not very optimized, so I wrote my own. He was also using his own version of the Delicious button and is slightly outdated. This following code uses the button from the Delicious site that pops up a javascript window and includes my code to add the bookmark count next to the link.

In your blogger template, place the following code somewhere inside the head tag:

<script type='text/javascript'>
  function print_delicious_data(data){
    var urlinfo = data[0] || {};
    if (!urlinfo.total_posts) return;
    document.write(urlinfo.total_posts + " save");
    if(urlinfo.total_posts!=1) document.write("s");
  }
</script>

Put the rest inside <div class="post-footer">

<a expr:href='"http://delicious.com/save?url="+data:post.url + "&amp;title=" + data:post.title'
   expr:onclick='
    "window.open(
      \"http://delicious.com/save?v=5&amp;noui&amp;jump=close&amp;url=\" +
        encodeURIComponent(\"" + data:post.url + "\") + \"&amp;title=\" +
        encodeURIComponent(\"" + data:post.title + "\"),
        \"delicious\",\"toolbar=no,width=550,height=550\"
    ); return false;"'
>Bookmark this on Delicious</a>


<script expr:src='"http://feeds.delicious.com/v2/json/urlinfo?url=" +
data:post.url + "&amp;callback=print_delicious_data"'/>

Just for fun, here is the code I use for the Digg and reddit buttons.

<script type='text/javascript'>
  digg_url = '<data:post.url/>';
  digg_title = '<data:post.title/>';
  digg_topic = 'programming';
  digg_skin = 'compact';
  digg_window = 'new';
</script>
<script src='http://digg.com/tools/diggthis.js' type='text/javascript'/>

<script type='text/javascript'>
  reddit_url='<data:post.url/>';
  reddit_title='<data:post.title/>';
  reddit_newwindow='1';
</script>
<script src='http://www.reddit.com/button.js?t=1' type='text/javascript'/>

Multiple asynchronous jQuery forms OR Making jQuery objects work for you

2008-11-13T17:30:00+00:00

There are times when I find the need to put multiple forms on a page. Lets say for example I have a couple of surveys for my users. I want them to be able to participate in all of the surveys, but I don’t want to make them hit the back button to fill out the subsequent forms. This, of course, can be done pretty easily with with ajax. Using both jQuery and the Forms Plugin, this is just a single line of javascript.

$("form").ajaxForm();

That was easy enough. All my forms will submit via ajax now. While this will work, it isn’t quite enough. Users like feedback, and right now they wont even know that anything happened.

I like to use a submit button (html button as opposed to input type=”submit”) that has an image that changes to a spinner when the form is submitted. I also disable the buttons while the form is submitting. (There is a nice tutorial on styling buttons this way.)

The forms plugin gives us a nice beforeSubmit() function. It is similar to jQuery’s ajax beforeSend(), but provides different input parameters. I will take advantage of the second parameter here – the form object. This will swap out the submit button’s image with a spinner image and disable the buttons before the XHR request is sent.

var spinner = new Image();
spinner.src = "/path/to/spinner.gif";

$("form").ajaxForm({
  beforeSubmit: function(data, $form, opts){
    $form.find('button').attr('disabled', 'disabled');
    $form.find('button img').replaceWith($(spinner).clone());
  }
});

Okay, great. We don’t want to spin forever though, so we should change the images back when the request is done processing. I reset my buttons in the complete callback. You can use the success callback if you’d like, but if the form submission returns an error code then your button will be spinning and disabled forever.

The problem with both the ‘complete’ and the ‘success’ callbacks is they don’t give you the form object like the beforeSubmit callback does. If you just have one form on the page this isn’t really a big issue because you can just use jQuery to select the form again via $('form') within the callback. I have multiple forms on my page though. If the second form is submitted before the first form completes, then both forms will be reset when the first one finishes.

I need a way to know which form submission is completing inside the callback. I can do this by storing the form in the jQuery.ajax() options object. Inside any of the callback functions ‘this’ refers to the options object, so it actually quite convenient.

var spinner = new Image();
spinner.src = "/path/to/spinner.gif";

$("form").ajaxForm({
  beforeSubmit: function(data, $form, opts){
    this.$form = $form;
    this.$buttonImage = $form.find('button img');
    $form.find('button').attr('disabled', 'disabled').find('img').replaceWith( $(spinner).clone() );
  },
  complete: function(){
    this.$form.find('button').removeAttr('disabled').find('img').replaceWith(this.$buttonImage);
  }
});

I do think it is a little shady to be modifying jQuery objects. If for some reason they decide to add a $form property in the future and I upgrade my jQuery then obviously my code will overwrite that property. I think the benefits outweigh the risk though.

In my fully tricked out forms, I’ll even add my own functions to the object to make things easier. Here is code for some forms that have both a save and cancel button and has server side validations. Fields that are invalid are passed back in the response. This will also display both success and error messages after submission. I have our server generate a 422 (Unprocessable Entity) on invalid forms. jQuery doesn’t automatically extract the response data for in the error call back, but it provides the XHR object so it is pretty easy to extract it myself.

$("form").ajaxForm({
  resetForm: true,
  beforeSubmit: function(data, $form){
    this.$form = $form;
    this.$saveButton = $form.find('button.save');
    this.$saveButtonImage = this.$saveButton.find('img');
    this.$saveButton.attr('disabled', 'disabled').find('img').replaceWith($(loadingImage).clone());
  },
  clearMessages: function(){
    this.$form.find('.invalid').andSelf().removeClass('invalid');
    this.$form.find('.errors, .success').remove();
  },
  complete: function(){
    this.$saveButton.removeAttr('disabled').find('img').replaceWith(this.$saveButtonImage)
  },
  success: function(){
    this.clearMessages();
    this.$form.prepend('&lt;div class="success"&gt;&lt;p class="heading"&gt;Your submission has been recorded.&lt;/p&gt;&lt;/div&gt;');
    this.$form.find(".success")[0].scrollIntoView();
  },
  error: function(XHR){
    this.clearMessages();
    var $form = this.$form;

    if(XHR.status == 422){
      var data = eval("(" + XHR.responseText + ")");

      if(typeof(data['invalid_fields']) != "undefined") {
        var error_box = '&lt;div class="errors"&gt;&lt;p class="heading"&gt;There were errors processing your submission.&lt;/p&gt;&lt;ol&gt;';

        $.each(data['invalid_fields'], function(id, errors){
          $form.find('.form_field_'+id).addClass('invalid');
          $.each(errors, function(){
            error_box += '&lt;li&gt;' + this + '&lt;/li&gt;';
          });
        });

        error_box += '&lt;/ol&gt;&lt;/div&gt;';
        $form.prepend(error_box);
        $form.find(".errors")[0].scrollIntoView();
      }
    }
  }
});