For a few months now, I have been working on a project that employs the GitHub APIs. The application I’m building is completely client-side and provides a feature to upload files to a repository on GitHub. To simplify the interaction with the API, I’m using a library called Github.js.
Developing this feature has not been straightforward since the documentation of the library lacks examples of this type, or even mention if it’s possible at all. Moreover, I wasn’t able to find any example on the web. Apparently, this functionality is of interest for many people as demonstrated by this question on StackOverflow and this issue opened on the Github.js’ repository. With a bit of study of the library’s source I was eventually able to develop the functionality, so I thought to write a post to share my experience and the solution I came up with.
In this article, I’ll explain how to upload any file on a repository on GitHub using Github.js and discuss a couple of major issues you might face. If you want to see (a better version of) the code developed for this article in action, you can browse and download it from the repository I’ve created on GitHub.
Saving text-based files with Github.js
Github.js exposes different methods that make the interaction with the GitHub API easier. To save a text-based file on GitHub, such as Markdown or HTML, the code needed is quite short and simple.
To start, you have to download and include Github.js in your project. Once done, you create a new instance of the Github object exposed by the library and create an object representing the repository you want to work with. Finally, you create a new file on the repository (or update it if the file already exists) with the content you want using a method of the library called write(). In the Git parlance, creating or updating a file means commit. Therefore, you also need to specify a commit message.
The code that implements these steps should look like the following snippet:
// Creates a new instance of the Github object exposed by Github.js
var github = new Github({
username: 'YOUR_USERNAME',
password: 'YOUR_PASSWORD',
auth: 'basic'
});
// Creates an object representing the repository you want to work with
var repository = github.getRepo('A_USERNAME', 'A_REPOSITORY_NAME');
// Creates a new file (or updates it if the file already exists)
// with the content provided
repository.write(
'BRANCH_NAME', // e.g. 'master'
'path/to/file', // e.g. 'blog/index.md'
'THE_CONTENT', // e.g. 'Hello world, this is my new content'
'YOUR_COMMIT_MESSAGE', // e.g. 'Created new index'
function(err) {}
);
At the moment, it isn’t possible to create or update more than one file with one commit using the GitHub API, thus using Github.js. So, every time you call the write() method, a new commit will be created.
The GitHub API accepts data that represent the content of the file in Base64. In the previous example I was able to provide a simple string because when write() Github.js converts it into the appropriate format (Base64) using the window.btoa() method. As you’ll learn in the remainder of the article, this feature will turn into an issue when dealing with non text-based files, but this won’t be a problem with the solution I’ll explain.
Uploading files with Github.js
To let your web application upload files on GitHub with Github.js, you need a way to select these files. Luckily, HTML has a native element designed for this purpose:
<input type="file" />
In case you need to upload more than one file, you can use the multiple attribute as shown below:
<input type="file" multiple />
With this element in place, you’re able to select one or more files but nothing more than that happens. So, you have to write the code that reads the content of the selected files and upload them on GitHub. Let’s start with the first task.
Reading files with JavaScript
The first task can be achieved by employing FileReader and its readAsDataURL() method. readAsDataURL() accepts a Blob or a File object and gives you the content as a Base64 encoded string. In the previous sentence I’ve used the word gives instead of return on purpose. In fact, the method itself doesn’t return the content (it returns undefined) because the read operation is asynchronous. To retrieve the content you have to listen for the loadend event which is fired when the read operation is completed.
To better understand how this mechanism works, let’s create a function called readFile() that accepts a File instance and reads its content. To avoid the issue known as callback hell, the function will return a Promise. If you need a primer on Promises, I suggest you to read the article JavaScript Promises, There and back again by Jake Archibald.
The readFile() function resolves the Promise if the read is successful or reject it if an error occurs. When the Promise is resolved, the function provides an object with two properties: filename and content. The former specifies the name of the file. The latter defines the binary representation of the file’s content, with the information about its mime type and its encoding removed. You may wonder why the content is binary encoded if readAsDataURL() returns the string in Base64. The reason is that we have to convert it to a binary encoding, using the window.atob() method, to compensate the opposite conversion that Github.js does on our behalf when calling write(). If you don’t do that, Github.js will send to the GitHub API a string that is double encoded in Base64 (i.e. Github.js encodes in Base64 a string that is already in Base64). As a result, if you try to open the file in your browser via the GitHub web interface or download it, you will receive an error because the file isn’t encoded correctly.
The code that implements the above description is shown below:
function readFile(file) {
return new Promise(function (resolve, reject) {
var fileReader = new FileReader();
fileReader.addEventListener('load', function (event) {
var content = event.target.result;
// Strip out the information about the mime type of the file and the encoding
// at the beginning of the file (e.g. data:image/gif;base64,).
content = atob(content.replace(/^(.+,)/, ''));
resolve({
filename: file.name,
content: content
});
});
fileReader.addEventListener('error', function (error) {
reject(error);
});
fileReader.readAsDataURL(file);
});
}
The readFile() function enables us to read one or more files, so we are now ready to upload them. But before discussing how to do that, we have to create a support function to overcome an issue of the GitHub API. Let’s learn more.
Promisifying Github.js
The application I’m working on requires to upload one or more files at a time, no matter how many commits are created. In my first attempts, I thought to speed up the process by executing all the calls to write(), one for each file, in parallel. However, the process was always failing with a weird error message. While doing a bit of research to understand what the cause could be, I stumbled upon this question on StackOverflow, and specifically this comment:
I’m guessing you’re hitting a corner case with the API due to these sequential writes that are happening one after another. Can you please try putting a sleep in your script so that you wait for a second or two between calls? That should resolve the issue. We’re working on fixing this on our end, but I can’t make any Promises about when this will be resolved. Let me know if that doesn’t help.
Now, having a sleep in between the calls was a no go for me. So, I decided to promisify the write() method so that I could execute all the calls in sequence without having to resort to any sleep approach. This decision turned out to be a winner.
The function to promisify write() is reported below:
function saveFile(data) {
return new Promise(function(resolve, reject) {
data.repository.write(
data.branchName,
data.filename,
data.content,
data.commitTitle,
function(err) {
if (err) {
reject(err);
} else {
resolve(data.repository);
}
}
);
});
}
With this code in place, I can introduce you to the last function needed.
Uploading files
To complete the feature we’re discussing, all we need is a function that accepts an array (or array-like object) of files to read and a commit message. I’ll call such function uploadFiles(). It’ll return a Promise that is resolved when all the files have been uploaded correctly or rejected in case of error. The body of the function is short but terse, and might be a bit complicated to understand. So, before showing the code let’s discuss it in depth.
The first operation to perform is to read all the files provided and store the resulting Promises into an array. The read operations are executed in parallel to improve the application’s performance. To complete this step we’ll use the readFile() function discussed earlier and map(). To move to the next step, we have to wait until the read of all the files is completed. To achieve this goal, we’ll employ Promise.all() and pass to it the array of Promises we created.
Once all the Promises are resolved, the function can start sending the data to GitHub using saveFile(). As I mentioned before, unfortunately the requests must be sent in sequence and we can’t upload all the files at once with a single commit. This means that a new request has to start when the previous one is resolved. For this reason, we have to create a chain of then() calls. In addition, for each call to then(), a new commit will be created. The are many ways we can achieve this goal, the simplest of which is to use a for loop or a similar construct. Personally, I preferred to use a functional approach that employs reduce(). The chain of Promises created is the returned value of the function.
The code of the uploadFiles() function is listed below:
function uploadFiles(files, commitTitle) {
// Creates an array of Promises resolved when the content
// of the file provided is read successfully.
var filesPromises = [].map.call(files, readFile);
return Promise
.all(filesPromises)
.then(function(files) {
return files.reduce(
function(promise, file) {
return promise.then(function() {
// Upload the file on GitHub
return gitHub.saveFile({
repository: gitHub.repository,
branchName: config.branchName,
filename: file.filename,
content: file.content,
commitTitle: commitTitle
});
});
},
Promise.resolve()
);
});
}
The caller of uploadFiles() can use the returned Promise to show a successful message to the user once the whole process is completed, or an error message if an issue occurs at any stage. An example is shown below:
var files = document.getElementById('file').files;
var commitTitle = 'Files uploaded';
uploadFiles(files, commitTitle)
.then(function() {
alert('Your file has been saved correctly.');
})
.catch(function(err) {
console.error(err);
alert('Something went wrong. Please, try again.');
});
Done! You’re now ready to upload as many files as you want on GitHub via its API using Github.js.
Conclusions
In this tutorial I’ve explained how to use Github.js to upload files on GitHub via its APIs. Finding the solution to this problem wasn’t straightforward and I had to dig into the library’s source to fully understand what was going on. Overall it has been a good experience that let me understand a couple of important issues to take into account. Specifically, I learned that behind the scene Github.js converts a string into its Base64 equivalent using the window.btoa() method. I also learned that the GitHub API doesn’t allow you to create or save multiple files with one commit, and that it has an issue with sequential writes that happens quickly one after another.
If you want to see (a better version of) the code developed for this article in action, you can browse and download it from the repository I’ve created on GitHub.