08 June 2023 JD

Uploading a large amount of small files to S3 quickly

TL;DR

Make sure you have Python 3 installed and run:

# install pip install boto pip install python-magic brew install libmagic // this one got me for a second as I'm running an M1 Mac # get the script and allow it to be executed curl https://gist.githubusercontent.com/darbio/8429cb82c00b7d4a2d9069a10f9f08bd/raw/28da5aba3935b05bde3e6d6b96b110b5e8bd3d2b/s3-parallel-put.py --output s3-parallel-put.py chmod +x ./s3-parallel-put.py # execute ./s3-parallel-put.py --bucket=BUCKET_NAME ./source-directory 

The detail

I recently had a solution which required the generation...

Continue Reading →

07 May 2023 JD

SwiftUI binding in UIKit with UIHostingController and ObservableObject

Often I have found myself wanting to use SwiftUI data bindings in Views hosted via UIHostingController in UIKit. A great example of this is when you want to use a SwiftUI View in a MKAnnotationView which has a data binding in the SwiftUI View.

Obviously, UIKit and SwiftUI don’t work with the @State of @Binding property wrappers, however @ObservedObject still works well and changes made on the UIKit code...

Continue Reading →

12 January 2016 JD

ThreadPool exceptions in IIS Express

When debugging a WebAPI application recently which used an asynchronous library, I came accross a situation where the application kept on throwing the following InvalidOperationException:

There were not enough free threads in the ThreadPool to complete the operation.

Running the following code from a WebAPI controller:

Results in the following:

workers = 2

completions = 2

This is too low to run any other async...

Continue Reading →

04 January 2016 JD

Event-based architecture using AWS (SNS and SQS) and C#

Introduction

The following code is an example of an event based architecture using an SNS topic per event. It consists of:

  • Producer - A system which produces events to an SNS topic.
  • Consumer - A system which consumes messages placed onto an SQS queue by an SNS topic.

The two events we are interested in are:

  • INCIDENT_CREATED
  • INCIDENT_UPDATED

We will create consumers...

Continue Reading →

10 November 2014 JD

Lookup lists (using MongoDB)

Darb.io Lookups (in MongoDB)

An example of how to do lookups in MongoDB, without persisting the data or having to maintain relationships etc. in your services.

Background

In an object model we often want to specify a domain type (as in ‘domain classification’, not object type) to an object.

Example

Take the example of an Organization. In our domain, Organizations have a name, and an organization type (classification). The organization...

Continue Reading →

30 June 2014 JD

WebAPI and ADFS as external login provider

Setting up ADFS

For these steps, I presume that you are running ADFS 3 on Windows Server 2012 R2 and that it has been already installed and configured.

Setting up the Relying Party Trust

  1. Open ADFS Managememt
  2. Expand ‘Trust Relationships’ and click on ‘Relying Party Trusts’.

http://i.imgur.com/HeGm5yl.png

  1. Click on ‘Add Relying Party Trust’ in the right hand panel.

http://i.imgur.com/o22tVfW.png

  1. Click ‘Start’ and...

    Continue Reading →

01 June 2014 JD

API best practice design

Here follows an aggregation of some generic rules for generating a Web API to program against. Based upon a number of references, supplied at the bottom of this post.

API design rules

Use REST verbs

Use HTTP verbs which make sense from the perspective of the API consumer:

  • GET will retrieve an resource
  • POST will create a resource
  • PUT will update a resource
  • PATCH...

    Continue Reading →

26 March 2014 JD

Architecting on AWS - Day 3, Sydney

These are my notes from day 3 of the Amazon ‘Architecting on AWS’ course which was run in Sydney in March 2014. The trainer/presenter was John Rotenstein from AWS.

Simple Queue Service

  • Regionally based
  • Automatically replicated throughout AZs
  • Concept of dead letter queues
  • Eventually consistent
  • FIFO, kind of…
  • “Visibility Timeout” is the invisibility timeout
    • Can be overridden in code
  • Messages can be retained for up to 14 days
  • Messages are...

    Continue Reading →

26 March 2014 JD

Architecting on AWS - Day 2, Sydney

These are my notes from day 2 of the Amazon ‘Architecting on AWS’ course which was run in Sydney in March 2014. The trainer/presenter was John Rotenstein from AWS.

Route 53

  • AWS DNS provider
  • Split DNS? Is this possible?
  • Subdomains can CNAME to an aws address (e.g. assets.s3.aws.com)

Routing

  • Round robin
    • Weighted
      • 0 will stop traffic going to that address
    • Equal
  • Latency based

26 March 2014 JD

Architecting on AWS - Day 1, Sydney

These are my notes from day 1 of the Amazon ‘Architecting on AWS’ course which was run in Sydney in March 2014. The trainer/presenter was John Rotenstein from AWS.

Best Practices

Design for failure

  • Design for failure, nothing fails. E.g. Aviation, seat belts, “black box”
  • “Everything fails, all the time”
  • Avoid single points of failure

Loose coupling sets you free

29 October 2012 JD

XmlSerialization considerations (on MonoTouch)

I recently was making an app which needed to store data on the client. The obvious choices were:

  1. Local file storage (e.g. serialise a class to XML and store it in Documents)
  2. Use an SQLite database to store the data and read/write when needed

I have done both before, and generally find that the XML storage is a quick and easy method of storing data...

Continue Reading →

11 August 2012 JD

FDI Calculator App in iOS AppStore

My first (personally funded and created) app is now up in the Apple App Store.

The app is a tool for:

  1. Fire Danger Index calculator
  2. Fire Danger Rating information
  3. Drought Factor calculator

From the AppStore app description:

FDI Utility is an application to calculate the Forest Fire Danger Index (FFDI), Grass Fire Danger Index (GFDI) and Drought Index (KBDI). It displays the results as the Fire Danger Index (FDI) and also...

Continue Reading →

25 July 2012 JD

Run tests from networked location

I use Parallels on OSX to run Visual Studio inside a VM.

With a test project I had written, I was getting the following exception thrown

Unit Test Adapter threw exception: URI formats are not supported.. Which was highly annoying!

After some googling, I found the solution:

  • Double click on Local.testsettings which is under the Solution Items of Project 1
  • Test Settings window is displayed. Click on the...

    Continue Reading →

25 July 2012 JD

Compile mono from source on OSX

The following is how I compiled mono latest from source:

  1. Install MacPorts from http://macports.org
  2. Install gettext
    sudo port install gettext
  3. Locate your current mono environment. It is usually at:
    /Library/Framework/Mono.framework/Versions/Current
  4. Download the mono source from GitHub master branch
    git clone https://github.com/mono/mono.git
  5. Run autogen with your OSX version prefix
    ./autogen.sh --prefix=/Library/Framework/Mono.framework/Versions/Current
  6. Run make
    sudo make
  7. Optionally, you can run the tests
    sudo make check
  8. Run make install
    sudo...
    					

    Continue Reading →

11 July 2012 JD

Mercurial Branching

We are currently going down the path of switching from our huge monolithic SVN repository, to a number of product based Mercurial repositories. Historically, I have used FogBugz and Kiln, but this time we have decided to give the JIRA stack a whirl - mainly for GrassHopper, as one of the project managers here wanted to give it a go.

In the old SVN repo, instead of branching and tagging, we copied files to “Release...

Continue Reading →

21 June 2012 JD

Microsoft Azure DevCamp in Sydney

Some may know that I went along to the Azure DevCamp in Sydney yesterday.

I have some plans for Azure, and hadn’t really had a chance to have a play with it before yesterday… Well.. I was impressed, and had a lot of fun!

The new portal is pretty schmick! And makes it easy to create:

  • Websites
  • Cloud Services
  • Storage
  • SQL DB's

Best thing is that, during preview, it is free for 90...

Continue Reading →

23 May 2012 JD

Modal loading dialog in MonoTouch

Update: I was using this in a project which called the UILoadingView class from an anonymous delegate and kept coming up against the dreaded SIGSEGV error… Well it was a silly mistake on my behalf. I forgot to wrap the DismissWithClickedButtonIndex(0, true); in BeginInvokeOnMain (...). Code updated below.

As some of you may know, I work as a Solutions Architect at IQX in Sydney - a company specialising in SAP integration with,...

Continue Reading →

12 May 2012 JD

Resources in MonoTouch assembly

In my experience, I found that adding resources into a MonoTouch project can be slightly confusing - if you have not done it before.

I had a requirement to embed images into a project… easy you may say. Add a solution folder into the solution, add your images into the folder and give them a build action of ‘Content’. Duh… Why bother with a blog post? I hear the crowd mutter.

Well… It on the...

Continue Reading →

21 March 2012 JD

jQuery Validate and MaxLength

A quick one… Just had a situation where we were adding the maxlength attribute to a html input, and then trying to use the excellent jQuery Validate plugin to validate the form… Resulting in an invalid form! Doh!!

Seems it is a bug…

Here is the fix.

Doh! Hopefully fixed soon.

Happy coding!!

Continue Reading →

05 March 2012 JD

Handling .Net DateTime in jQueryUI DatePicker using KnockoutJS

In a recent project we were using KnockoutJS to bind a .Net model to a JavaScript ViewModel to a HTML View.

The View has a number of jQuery UI controls on it, including the infamous the DatePicker.

The model had a number of ‘complex’ types on it, however the type which gave us the most headache was the ‘simple’ .Net CLR DateTime type.

When the JavaScriptSerializer serializes a .Net...

Continue Reading →

24 February 2012 JD

Sending an email with an attachment from Workflow Foundation

The problem:

The SendEmailActivity in WF sucks… because you can’t send an email with attachments from it.

I want to be able to send an email from a SharePoint WorkFlow, using the SP Outbound Mail Service and adding some attachments to the message.

The workaround:

Use the .Net MailMessage class (System.Net.Mail) and a Code activity to send the email:

Non-SharePoint example:

SharePoint example:

...

Continue Reading →

24 February 2012 JD

500 Internal Server Errors when using jQuery.ajax in MOSS SharePoint 2007 SP1

I have recently been architecting a SharePoint forms solution which uses jQuery.ajax To communicate, using JSON, with a .Net 2.0 web service - hosted in the \_vti\_bin / ISAPI folder of a SharePoint host.

For various reasons we have been developing in SharePoint 2010, and it has worked fine in our dev environments. Today we did our first preliminary deployment of the solution into the client’s development environment…

After some fun and games...

Continue Reading →