How to configure OnMyCommand to generate a progress bar for curl
Also: How to get make my boss happy as a clam
One of the initial capabilities I have built into my company's server application (a streaming media content management system) is RESTful content CRUD (Create, Retrieve, Update, Delete) access. Our system's web-based administration interface is written in Flex, and unfortunately Adobe's Flash plug-in does not support HTTP PUT (among other RESTful actions), so all of our content CRUD actions occur either through our client side application or manually.
"Manually" means me, ssh and sftp. Over the past several months I have been increasingly micro-tasked by my colleagues with uploading special content file updates that we call "shared players". This process, because of the Fort Knox-grade security Aspectra, our hosting company, is not particularly simple, and somewhat error prone because of all of the intermediate steps.
We need a better solution--something fast, secure, and (preferably) no Stu required! So, I could either jump into the administrative UI code (b.o.r.i.n.g--UI coding is for mere mortals), or I could come up with something kinda interesting.
To speed the process of late, I had been using curl at the command line to upload these files via HTTP. The syntax is pretty simple:
My efforts to get my colleagues using the above failed miserably. (Most people are scared of CLI, it seems.) So, an OS X shell integrated right-click "Upload this file to the CMS" thingy seemed like a great idea.
A quick Google search of the Internets introduced me to the free OnMyCommand. Not only does OMC allow me to create right-click shell functionality, but also "Droplets", which are self contained applications that can be used without installing OMC. Setting up my own curl command was a piece of cake--I had it running inside an hour. The documentation and examples are perfect for this. OMC has a neat little feature where you can capture the output from a CLI command and use it to create a progress bar! For file uploads that is a very useful feature--I spent the next four hours putting icing on the cake. The documentation here is a bit more sparse (a newish OMC feature apparently), examples even fewer, and the OMCEdit GUI's "progress" setup a bit clunky.
The result rocks my geek soul. Here is how the tool is used: Right-click the file one wants to upload, and...
(Yes, I too find the menus a bit too verbose. I'll look into that later.)
(OMC v2.1 (2008-11-28) was used here. The software is under active development and YMMV substantially.)
Immediately afterward the progress bar pops up showing the progress, including percent uploaded, bytes uploaded and average upload speed.
Fracking groovy, no? Here is how the progress bar was configured (after some blood, sweet and tears.) The OMCEdit helps to start things out, and this is what works:
Figuring out what worked and what did not was painful, but are easily traced to two points:
This parses the curl output status line into the subgroups that we need for the status line:
$1: percent finished
$2: size uploaded so far
$6: size to upload
$8: average upload rate
UPDATE: A much shorter regular expression thanks to a stackoverflow.com user. Details in this post
The second issue is dealt with very easily: append &2>1 to the curl command line argument configured in OCM. I think it would be better if the OMC progress bar configuration allowed one to specify as much, but it's not my awesome free software project.
And there we go, OMC progress bar for curl. I've most certainly skipped over details that seemed trivial at the time. So, if you have questions about anything please do comment with the blog system, or even email me (stu (aatt) dietpizza.ch) where you are sure to get a response.
"Manually" means me, ssh and sftp. Over the past several months I have been increasingly micro-tasked by my colleagues with uploading special content file updates that we call "shared players". This process, because of the Fort Knox-grade security Aspectra, our hosting company, is not particularly simple, and somewhat error prone because of all of the intermediate steps.
We need a better solution--something fast, secure, and (preferably) no Stu required! So, I could either jump into the administrative UI code (b.o.r.i.n.g--UI coding is for mere mortals), or I could come up with something kinda interesting.
To speed the process of late, I had been using curl at the command line to upload these files via HTTP. The syntax is pretty simple:
curl -T file.txt -u user:pass server.com/path/file.txt My efforts to get my colleagues using the above failed miserably. (Most people are scared of CLI, it seems.) So, an OS X shell integrated right-click "Upload this file to the CMS" thingy seemed like a great idea.
A quick Google search of the Internets introduced me to the free OnMyCommand. Not only does OMC allow me to create right-click shell functionality, but also "Droplets", which are self contained applications that can be used without installing OMC. Setting up my own curl command was a piece of cake--I had it running inside an hour. The documentation and examples are perfect for this. OMC has a neat little feature where you can capture the output from a CLI command and use it to create a progress bar! For file uploads that is a very useful feature--I spent the next four hours putting icing on the cake. The documentation here is a bit more sparse (a newish OMC feature apparently), examples even fewer, and the OMCEdit GUI's "progress" setup a bit clunky.
The result rocks my geek soul. Here is how the tool is used: Right-click the file one wants to upload, and...
(Yes, I too find the menus a bit too verbose. I'll look into that later.)
(OMC v2.1 (2008-11-28) was used here. The software is under active development and YMMV substantially.)
Immediately afterward the progress bar pops up showing the progress, including percent uploaded, bytes uploaded and average upload speed.
Fracking groovy, no? Here is how the progress bar was configured (after some blood, sweet and tears.) The OMCEdit helps to start things out, and this is what works:
Figuring out what worked and what did not was painful, but are easily traced to two points:
- my regular expressions knowledge is not what is could be
- OMC's progress feature expects input from std in, and curl (somewhat unusually) sends status information to std err.
^\s*([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)\s+([^ ]+)This parses the curl output status line into the subgroups that we need for the status line:
$1: percent finished
$2: size uploaded so far
$6: size to upload
$8: average upload rate
UPDATE: A much shorter regular expression thanks to a stackoverflow.com user. Details in this post
The second issue is dealt with very easily: append &2>1 to the curl command line argument configured in OCM. I think it would be better if the OMC progress bar configuration allowed one to specify as much, but it's not my awesome free software project.
And there we go, OMC progress bar for curl. I've most certainly skipped over details that seemed trivial at the time. So, if you have questions about anything please do comment with the blog system, or even email me (stu (aatt) dietpizza.ch) where you are sure to get a response.