{"id":686,"date":"2015-06-27T05:27:37","date_gmt":"2015-06-27T00:27:37","guid":{"rendered":"http:\/\/52.24.147.87\/?p=455"},"modified":"2018-12-20T08:47:01","modified_gmt":"2018-12-20T03:47:01","slug":"adding-gitlab","status":"publish","type":"post","link":"http:\/\/www.stonemillkids.com\/index.php\/2015\/06\/27\/adding-gitlab\/","title":{"rendered":"GitLab &#8211; Setting up a Self-Hosted git Server"},"content":{"rendered":"<p>Before I setup GitLab, I did a fair amount of investigation. I found that GitHub and GitLab were common tools that offered, in my opinion, better git integration than Mantis (which I had some previous experience with).\u00a0 As such, I decided to switch off of Manius.\u00a0 After considering GitHub, I decided to use GitLab.\u00a0 There were multiple reasons for this, but the main one was I could host GitLab locally.\u00a0 And while GitHub is free for open source or non-commercial use, the licensing seemed more restrictive.\u00a0 I like the idea of being able to host my own repository and charge for my code in the future, even it that is not a goal today.\u00a0\u00a0 Other reasons to move off of Mantis were:<\/p>\n<ul>\n<li>GitLab uses git, which I decided to be my Version Control System<\/li>\n<li>It has web integration.\u00a0 This means that I can view the changes and track what I am doing on the web.<\/li>\n<li>It has a basic ticketing system.\u00a0 This means I can put my requirements, bugs, etc. into a backlog.<\/li>\n<li>It is free<\/li>\n<li>It supports multiple users.\u00a0 This means as the project grows, I&#8217;ll be able to use the same software.<\/li>\n<\/ul>\n<p>There are a fair amount of documentation on GitLab.\u00a0 I&#8217;m not going to repeat it all, but give the links that I found to be chronologically useful.<\/p>\n<p>Also they are very good at maintaining their documentation.\u00a0 I revisited many of these links five years after my post and found them still working.<\/p>\n<p><strong>Setting Up<\/strong><\/p>\n<p>There are three ways to setup GitLab.\u00a0 One is installing each individual component from source,\u00a0another is setting up an\u00a0entire AWS, bitnami, or other &#8220;total&#8221; system install.\u00a0\u00a0The last one, and the one that I\u00a0went with, was installing\u00a0an Omnibus install on an existing Linux Server.\u00a0 The this <a href=\"https:\/\/about.gitlab.com\/installation\/\">link <\/a>for full detail.<\/p>\n<p><em>Word of warning:\u00a0 Omnibus is not cheap on Memory usage (This will come up later).\u00a0 It may be better to use something like Bitnami, or another prebuilt image on a\u00a0dedicated\u00a0server if you have the options.<\/em><\/p>\n<p>Following the links above, I installed an rpm<\/p>\n<p><a href=\"https:\/\/packages.gitlab.com\/gitlab\/gitlab-ce\/packages\/el\/7\/gitlab-ce-7.12.0~omnibus.1-1.x86_64.rpm\">https:\/\/packages.gitlab.com\/gitlab\/gitlab-ce\/packages\/el\/7\/gitlab-ce-7.12.0~omnibus.1-1.x86_64.rpm<\/a><\/p>\n<p>The complete instructions are here:<\/p>\n<ol>\n<li><a href=\"https:\/\/gitlab.com\/gitlab-org\/omnibus-gitlab\/blob\/629def0a7a26e7c2326566f0758d4a27857b52a3\/README.md\">https:\/\/gitlab.com\/gitlab-org\/omnibus-gitlab\/blob\/629def0a7a26e7c2326566f0758d4a27857b52a3\/README.md<\/a><\/li>\n<\/ol>\n<p>I ran out of memory reconfiguring, so stopped a couple of processes, then was successfully able to complete the install.<\/p>\n<p>After install, and a reboot.\u00a0 The server showed up on the default port.<\/p>\n<p>So after the site was up and running, I needed to change the external port I was listening on because I was already running a web server on :80.<\/p>\n<p>editing \/etc\/gitlab\/gitlab.rb, by adding the port number at the end, like so:<\/p>\n<pre><code>external_url='http:\/\/myurl:myport'\r\n<\/code><\/pre>\n<p>and then running<\/p>\n<pre><code>sudo gitlab-ctl reconfigure<\/code><\/pre>\n<p>Note:\u00a0Stay away from port 8080.\u00a0\u00a0I changed\u00a0GiLab to run on a\u00a0different port because of a post I found here:<\/p>\n<p><a href=\"http:\/\/stackoverflow.com\/questions\/29320022\/gitlab-on-port-8080\"><u><span style=\"color: #0066cc;\">http:\/\/stackoverflow.com\/questions\/29320022\/gitlab-on-port-8080<\/span><\/u><\/a><\/p>\n<p>Edit file as described:<\/p>\n<pre>vi \/etc\/gitlab\/gitlab.rb\r\n<\/pre>\n<p>With the change, my file worked:<\/p>\n<pre>## https:\/\/gitlab.com\/gitlab-org\/omnibus-gitlab\/blob\/629def0a7a26e7c2326566f0758d4a27857b52a3\/README.md#configuring-the-external-url-for-gitlab\r\nexternal_url \"http:\/\/ip-172-31-78-13.us-west-2.compute.internal:XX\"\r\n<\/pre>\n<p>&lt;Where XX is my port&gt;<\/p>\n<p>&lt;and do not be afraid, my URL (ip-172-31-78-13.us-west-2.compute.internal)has also changed too&gt;<\/p>\n<p>After being confused as to why the site showed up as running, but did not seem to be accessible, I remembered that I needed to change the AWS firewall rules to my new port would be served through the AWS firewall.<\/p>\n<p>Next Step is to log into\u00a0the git server, make changes, etc.\u00a0 As for now.\u00a0 I&#8217;ll just turn it off and wait until\u00a0tomorrow.<\/p>\n<p>&#8230; the next day&#8230;<\/p>\n<p>So\u00a0this almost worked.\u00a0 Looking into the system, it seems that everything is sluggish, and sometimes GitLab completely\u00a0times out with\u00a0 a 404 error even though the server is running.<\/p>\n<p>With more research I found that it is documented as requiring 2GB of RAM min.\u00a0 AWS micro, only has 1GB Total.\u00a0 After rebooting and searching through logs I found out that the site is crashing due to being unable to allocate memory.<\/p>\n<p>As I continues to google, I found I could reduce the memory by changing the number of workers:<\/p>\n<p><a href=\"https:\/\/gitlab.com\/gitlab-org\/omnibus-gitlab\/blob\/master\/doc\/settings\/unicorn.md#unicorn-settings\">https:\/\/gitlab.com\/gitlab-org\/omnibus-gitlab\/blob\/master\/doc\/settings\/unicorn.md#unicorn-settings<\/a><\/p>\n<p>12\/19\/2019 &#8211; Update &#8211; by default unicorn now only uses 2 workers by default.\u00a0 I did a complete uninstall, reinstall, and imported all of my old git repositorieis (I did not port my GitLab &#8220;todos&#8221;) and it worked wonderfully as a new install\/upgrade to GitLab 11.5.4.<\/p>\n<p>Using ps -aux I saw that sidekiq is also running a ton of memory, but no amount of googling would turn up anything definitive on how to change the site&#8230; so I left it and googled other items.<\/p>\n<p>I was still crashing.<\/p>\n<p>Then I remembered about Linux&#8217;s swap space.\u00a0 I decided to add a swap partition on the HD.\u00a0 Keep in mind this is the HD that is allocated with the t2.micro (8GB).<\/p>\n<p>I created a swap partion<\/p>\n<pre>bash$\u00a0 dd if=\/dev\/zero of=\/mnt\/swap bs=1M count=2000\r\nbash$\u00a0 mkswap \/mnt\/swap\r\nbash$\u00a0 swapon \/mnt\/swap\r\nbash$\u00a0 free\r\ntotal\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 used\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 free\u00a0\u00a0\u00a0\u00a0 shared\u00a0\u00a0\u00a0 buffers\u00a0\u00a0\u00a0\u00a0 cached\r\nMem:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1020184\u00a0\u00a0\u00a0\u00a0 937176\u00a0\u00a0\u00a0\u00a0\u00a0 83008\u00a0\u00a0\u00a0\u00a0\u00a0 22388\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1664\u00a0\u00a0\u00a0\u00a0\u00a0 46616\r\n-\/+ buffers\/cache:\u00a0\u00a0\u00a0\u00a0 888896\u00a0\u00a0\u00a0\u00a0 131288\r\nSwap:\u00a0\u00a0\u00a0\u00a0\u00a0 2097148\u00a0\u00a0\u00a0\u00a0\u00a0 0 2097148\r\n<\/pre>\n<p>Keep in mind that the bs=1M actually means 1024, so the count can be 2000 for a total of 2GB space.<\/p>\n<p>I also added swap to the \/etc\/fstab so this will happen automatically on reboot.<\/p>\n<pre>$ more \/etc\/fstab\r\n#\r\nLABEL=\/\u00a0\u00a0\u00a0\u00a0 \/\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ext4\u00a0\u00a0\u00a0 defaults,noatime\u00a0 1\u00a0\u00a0 1\r\ntmpfs\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/dev\/shm\u00a0\u00a0\u00a0 tmpfs\u00a0\u00a0 defaults\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0 0\r\ndevpts\u00a0\u00a0\u00a0\u00a0\u00a0 \/dev\/pts\u00a0\u00a0\u00a0 devpts\u00a0 gid=5,mode=620\u00a0 0\u00a0\u00a0 0\r\nsysfs\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/sys\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 sysfs\u00a0\u00a0 defaults\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0 0\r\nproc\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/proc\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 proc\u00a0\u00a0\u00a0 defaults\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0 0\r\n\/mnt\/swap\u00a0\u00a0 swap\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 swap\u00a0\u00a0\u00a0 defaults\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0 0\r\n<\/pre>\n<p>I rebooted and swap was till there (and being used)<\/p>\n<pre>$ free\r\ntotal\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 used\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 free\u00a0\u00a0\u00a0\u00a0 shared\u00a0\u00a0\u00a0 buffers\u00a0\u00a0\u00a0\u00a0 cached\r\nMem:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1020184\u00a0\u00a0\u00a0\u00a0 941076\u00a0\u00a0\u00a0\u00a0\u00a0 79108\u00a0\u00a0\u00a0\u00a0\u00a0 22388\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2872\u00a0\u00a0\u00a0\u00a0\u00a0 50060\r\n-\/+ buffers\/cache:\u00a0\u00a0\u00a0\u00a0 888144\u00a0\u00a0\u00a0\u00a0 132040\r\nSwap:\u00a0\u00a0\u00a0\u00a0\u00a0 2097148\u00a0\u00a0\u00a0\u00a0 124284\u00a0\u00a0\u00a0 1972864\r\n<\/pre>\n<p>After a conversation with AWS technical support, it seems I will be charged for the I\/O on the swap space.\u00a0 As such, I searched for a way to reduce it.\u00a0 Luckily the Linux kernel has an idea of how to adjust how the kernel prioritizes swap space.\u00a0 Here is a good article:<\/p>\n<p>https:\/\/en.wikipedia.org\/wiki\/Swappiness<\/p>\n<p>I used the option of editing the vi \/etc\/sysctl.conf file and adding a lines<\/p>\n<pre># set the swap preferences:\r\nvm.swappiness = 1\r\n<\/pre>\n<p>After rebooting and launching everything I ran top and logged into the gitlab site:<\/p>\n<p>It still swapped about 2MB of memory, but at least we were not crashing.<\/p>\n<p>I set my swappieness to\u00a01 because I do not want to use it often.\u00a0 I have an open question to AWS tech support to determine some additional ways to monitor the system.\u00a0 If I find I am using too much swap space I&#8217;ll move my tier up.\u00a0 However, as it sits today, I&#8217;m still on the free tier, so it is good enough for now.<\/p>\n<p>&#8230; or so I thought<\/p>\n<p>Just when I was ready to start I realized that I did not actually have a git server installed.<\/p>\n<p>using yum, I installed git on the server and while I was at it, I thought it a good idea to update all of my packages (because now I am actually ready to start).<\/p>\n<p>GitLab, ironically enough, was one of the items that got upgraded.\u00a0 However, it did not function after the update.\u00a0 Scratching my head I simply uninstalled, reinstalled, and reconfigured and everything worked.<\/p>\n<p>It should also be noted that my upgrade did NOT change my \/etc\/gitlab\/gitlab.rb settings.\u00a0 That means I did not need to do any extra effort.\u00a0 Later while searching for something else, I came across a &#8220;how to upgrade&#8221; page for gitlab.<\/p>\n<p>They suggested using &#8220;gitlab-crl upgrade&#8221;\u00a0 I was not sure if the errors were due to recently installing git, so I moved on without trying this feature.\u00a0 I am documenting it so I can come back to it when I need to do it again.<\/p>\n<p>&#8211; Update 5\/31\/16 &#8211; the :gitlab-crl upgrade&#8221; as well as the &#8216;yum update all&#8221; both seem to function and leave my custom settings.<\/p>\n<p>I logged into gitlab, created a project,\u00a0and started testing how\u00a0GitLab works.<\/p>\n<p>I simply created a &#8220;TestMe&#8221; project so I could play with the features without messing up my real project. It was simple enough I did not need to read the documentation, but I figured that for this post I needed to find a tutorial.\u00a0 Here is one that seemed to cover all the items I did on my own:<\/p>\n<p><a href=\"https:\/\/www.digitalocean.com\/community\/tutorials\/how-to-use-the-gitlab-user-interface-to-manage-projects\" target=\"_blank\" rel=\"noopener\">https:\/\/www.digitalocean.com\/community\/tutorials\/how-to-use-the-gitlab-user-interface-to-manage-projects<\/a><\/p>\n<p>So, I created a test project.\u00a0 I tired to check out the project, but it failed.\u00a0 GitLab reminded me<\/p>\n<p>&#8220;You won&#8217;t be able to pull or push project code via SSH until you <span style=\"color: #0066cc;\">add an SSH key<\/span> to your profile&#8221;<\/p>\n<p>It also pointed me to a set of instructions that explained how to set the SSH key up, so I will not repeat them here.<\/p>\n<p><em>An interesting item of note.\u00a0 I am using TortoiseGit for Windows.\u00a0\u00a0Logging in can be done from Windows using the same GitLab username\/password as was created on the main page.\u00a0 Once the key is established on the GitLab project, it is not needed locally on Windows, it is simply stored on the Server.<\/em><\/p>\n<p>&#8211;<\/p>\n<p>Now that I have a repository and ticketing system I can enter the requirements, and code to them &#8211; I will blog this in detail later.\u00a0 For now, I am done.\u00a0Wow that took longer than I thought.<\/p>\n<p>:wq!\u00a0 &lt;- (joke)<\/p>\n<p>Update: Two weeks later GitLab still uses enough memory that I am turning it off occasionally. I am still tweaking the configuration while trying to monitor the costs of the AWS server. More will follow(or not), but my next steps might be to edit my cron jobs so that GitLab restarts daily at 4:30AM.<\/p>\n<pre># free\r\n\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 total\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 used\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 free\u00a0\u00a0\u00a0\u00a0 shared\u00a0\u00a0\u00a0 buffers\u00a0\u00a0\u00a0\u00a0 cached\r\nMem:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1020184\u00a0\u00a0\u00a0\u00a0 951772\u00a0\u00a0\u00a0\u00a0\u00a0 68412\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1596\u00a0\u00a0\u00a0\u00a0\u00a0 40116\u00a0\u00a0\u00a0\u00a0\u00a0 43408\r\n-\/+ buffers\/cache:\u00a0\u00a0\u00a0\u00a0 868248\u00a0\u00a0\u00a0\u00a0 151936\r\nSwap:\u00a0\u00a0\u00a0\u00a0\u00a0 2097148\u00a0\u00a0\u00a0\u00a0 404872\u00a0\u00a0\u00a0 1692276\r\n\r\n#sudo gitlab-ctl restart\r\n\r\nok: run: logrotate: (pid 7831) 1s\r\nok: run: nginx: (pid 7834) 0s\r\nok: run: postgresql: (pid 7840) 0s\r\nok: run: redis: (pid 7848) 1s\r\nok: run: sidekiq: (pid 7855) 0s\r\ntimeout: run: unicorn: (pid 1730) 125458s, got TERM\r\n\r\n# free\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 total\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 used\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 free\u00a0\u00a0\u00a0\u00a0 shared\u00a0\u00a0\u00a0 buffers\u00a0\u00a0\u00a0\u00a0 cached\r\nMem:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1020184\u00a0\u00a0\u00a0\u00a0 653752\u00a0\u00a0\u00a0\u00a0 366432\u00a0\u00a0\u00a0\u00a0\u00a0 19996\u00a0\u00a0\u00a0\u00a0\u00a0 22268\u00a0\u00a0\u00a0\u00a0\u00a0 78120\r\n-\/+ buffers\/cache:\u00a0\u00a0\u00a0\u00a0 553364\u00a0\u00a0\u00a0\u00a0 466820\r\nSwap:\u00a0\u00a0\u00a0\u00a0\u00a0 2097148\u00a0\u00a0\u00a0\u00a0\u00a0 49032\u00a0\u00a0\u00a0 2048116<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Before I setup GitLab, I did a fair amount of investigation. I found that GitHub and GitLab were common tools that offered, in my opinion, better git integration than Mantis (which I had some previous experience with).\u00a0 As such, I decided to switch off of Manius.\u00a0 After considering GitHub, I decided to use GitLab.\u00a0 There [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":740,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[18,14],"tags":[24,42,31],"_links":{"self":[{"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/posts\/686"}],"collection":[{"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/comments?post=686"}],"version-history":[{"count":5,"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/posts\/686\/revisions"}],"predecessor-version":[{"id":933,"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/posts\/686\/revisions\/933"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/media\/740"}],"wp:attachment":[{"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/media?parent=686"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/categories?post=686"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.stonemillkids.com\/index.php\/wp-json\/wp\/v2\/tags?post=686"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}