Saturday, February 22, 2014

Setting Up a Maven Repository

By Avneet Mangat
01 Jun 2007 | TheServerSide.com

    1. Introduction

    Maven is a popular build tool available for java developers. One of the benefits of maven is that helps to reduce the duplication of dependent software libraries (jars) required to build an application. The maven suggested approach is to store all the software libraries in a remote store called a repository.
    Maven synchronizes with the public repository at ibiblio.org This repository is slow, unreliable and does not have the latest versions of some libraries(jars) available elsewhere. Libraries private to the organization cannot be uploaded there either. By setting up an internal maven repository, an organization can avail of the benefits of a maven repository and bypass some of the shortcomings of the repository at ibiblio.
    This article looks at some of the functionality that a maven repository should provide. Criteria for choosing a maven repository are listed. Steps involved in setting up a maven repository using Artifactory are explained. The process of setting up the repository is same for both Linux and Windows and the minor differences are highlighted in the article. Examples of maven ‘pom’ files which use this repository are shown. The article is illustrated with screenshots to guide the user in setting up a repository. Sample maven and artifactory configuration is shown.
    1.1 Background knowledge
    It is assumed that the reader is familiar with the following concepts and technologies:
    • Maven 2
    • JEE web servers deployment process (e.g. deploying web applications to Tomcat)
    • XML
    1.2 Purpose of maven repository
    The purpose of maven repository is to serve as an internal private repository of all software libraries used within an organization. Storing maven artefacts(jars and poms) in a dedicated maven repository is preferable to storing them in version control systems(such as CVS or Subversion) for the following reasons:
    • Libraries(jars) are binary files and don’t belong in version control systems which are better at handling text files
    • Keeps the version control database small
    • Checkouts, updates and other actions on the version control system will be quicker
    1.3 Advantages of having an internal private repository:
    • Reduced likelihood of version conflicts
    • Less manual intervention required for doing a build first time
    • A single central reference repository of all dependent software libraries rather than several independent local libraries
    • It is quicker to do a clean build when using an internal repository as maven artifacts are retrieved from a server on the intranet rather than the ibiblio server on the internet
    1.4 Types of maven repository
    • Local repository – exists on developers machine and is maintained by the developer. It is in sync with the maven repositories defined in the ‘settings.xml’ in their ‘~home/.m2’ folder. If no private remote internal repository is setup and listed in the pom.xml file or settings.xml file, then this local repository on developers machine is synchronized with the public maven repository at ibiblio.org
    • Private remote internal repository – This the repository which we will setup. We will change the maven pom.xml or settings.xml to use this repository
    • Public remote external repository – This is the public external repository at ibiblio.org. By default, maven synchronizes with this repository.

    2. Overview of maven repository

    2.1 Development environment without any repository
    2.2 Development environment with an organization wide internal remote repository

    3. Use cases for maven repository

    • Create 2 sub-repositories within the internal repository
      • ‘Ibiblio-cache’ for artefacts downloaded from ibiblio.org and available publicly. This would be synchronized with external repository from ibiblio
      • internal-maven-repository – for organizations internal artefacts. This is not synchronized with any external repository and contains artefacts which are unique to the orgranization
      • Alternatively, another sub-repository can be created for artefacts available publicly but not in ibiblio – e.g. later versions of some artefacts. This would not be synchronised with an external repository. We can call this ‘3rd-party’
    • Browse remote repository, preferably using a web browser
    • Search for artefacts in the repository
    • Download code from version control, change ‘settings.xml’ to point to the internal repository and do a clean build without any manual intervention
    • Install new version of an artefact
    • Bulk import artefacts into the repository
    • Bulk export artefacts from the repository
    • Backup the repository manually and setup task to do this automatically

    4. Criteria for choosing a maven repository implementation

    The ideal maven repository implementation should be:
    • Open source and free
    • Provide admin tools
    • Provide a repository browser – preferably web browser rather than desktop application.
    • Deployable in standard web server – e.g Apache or Tomcat
    • Ability to create, edit and delete sub repositories
    • Bulk import/export facility to move groups of artefacts into and out of the repository
    • Access control facility and anonymous read-only access
    • Easy to setup and use
    • Backup facility
    • Issue tracker, forums and other independent sources of information
    • Active community/developers so that the produce is enhanced and bugs fixed

    5. Comparison of various maven repositories

    Some of the popular open source and free maven repositories are:
    A comparison is shown below:
      Standard maven repositoryDead simple Maven Proxy (DSMP)ProximityArtifactory
      Admin toolsNoBasicYesYes
      Repository browserBasicNoYesYes
      Deployable in standard web server (e.g. Tomcat or Jetty)Yes, but not configurableNoYesYes(works in Tomcat and bundled with Jetty)
      Create, edit and delete sub repositoriesYesYesYesYes
      Bulk import/export artefactsNoNoNoYes
      Easy to setup and useYesNo, build from sourceYesYes(Ajax web UI)
      Backup facilityNoNoNoYes(using Java quartz API and ‘cron’ expressions)
      Issue tracker, forums and other sources of informationJira, IRCNoWiki and issues trackerJira issue tracker. Good documentation on site
    Following review of all products, Artifactory has all the criteria that we are looking for. Proximity maven repository also seems to have most of the features we are looking for. We will look at implementing the maven repository using Artifactory.
    Other points about Artifactory are:
    • Meets all our requirements
    • Apache 2.0 license
    • Runs on JDK 1.6 and Tomcat 6. Artifactory is deployed as a ‘war’ file in Tomcat.
    • Uses Derby DB to store artefacts. Data is stored in a published and well known format
    • All artefacts can be bulk exported to local repository and then imported into some other repository so it is easy to move artefacts from one implementation to another. It also makes upgrades of maven repository very easy
    • Compliant with Java Content Repository standard (JSR-170).
    • Uses Lucene as search engine. Repository is indexed and can be re-indexed
    • UI done using Ajax
    • Easy to customise the UI
    • Uses Jira as an issue tracker

    6. Setting up the maven repository

    6.1 Required software 6.2 Directory structure
    Down load and unzip artifactory. The directory structure is shown below:
    The folders are:
    • Backup – backup of repository. Backup policy can be setup using ‘cron’ expressions. Quartz scheduler is used to run the backup at the specified time. The backup interval is specified in the ‘ARTIFACTORY_INSTALLATION_FOLDER>/etc/artifactory.config.xml’
    • Bin – batch files used to run the included jetty web server
    • Data – Has derby database files. Everything in this folder can be deleted if you wish to start with a clean repository. In a new installation of artifactory, this folder is empty.
    • Etc – Configuration files for artifactory. Has ‘artifactory.config.xml’ and ‘jetty.xml ‘and ‘log4j.,properties’
    • Lib – has all the dependent jar files
    • Logs – log files
    • Webapps – has the ‘war’ file. This can be copied and installed in tomcat.
    6.2 Deploy in Tomcat 6
    Deploy the ‘war’ file in ‘_FOLDER>/webapp’ to ‘/webapps’. No tomcat configuration changes are required with jdk1.6 and Tomcat 6. Tomcat 6 should detect the web application and deploy it.
    Once the web applications is deployed, the web application needs this information:
    • Location of the database to store the artefacts
    • Location of the artifactory config xml file
    • Location of backup folder
    A single configuration is used to specify all 3. We only have to specify the location of the location of the artifactory installation folder during Tomcat startup and artifactory will be able to work out the rest. An alternative to this approach would have been to setup connection to the derby database using jdbc and to configure artifactory in the web application(by including artifactory.config.xml in the web application). However, this approach is simpler.
    The location of artifactory installation folder can be specified as a environment variable. For Linux, use ‘.bash’ script to export the location of the artifactory installation folder as shown below:
    export JAVA_OPTS = -Dartifactory.home=/home/amangat/artifactory-1.2.1-rc1
    For Windows, it can be added to Tomcat startup options as shown below:
    6.3 Setup the maven repositories
    A suggested approach is to create 3 repositories(or sub repositories) in our maven repository. They are:
    • Private-internal-repository: This repository contains artifacts which are used only within the organization. These are manually uploaded by the developer. This does not synchronize with remote repository with ibiblio as the artifacts in this repository(or sub-repository) are private to the organization.
    • 3rd-party: This contains artifacts which are publicly available but not in ibiblio repository. Example could be latest versions of libraries which are not yet available on ibiblio or jdbc drivers. This repository is not synchronized with ibiblio as ibiblio does not have these jars
    • Ibiblio-cache: This repository is synchronized with ibiblio repository and is a cache of the artifacts from ibiblio.
    This is configured in the _FOLDER>/etc/artifactory.config.xml’. The configuration to setup these 3 repositories is shown below:
    
        
        
        
          
              private-internal-repository
              Private internal repository
              true
              true
          
          
              3rd-party
              3rd party jars added manually
              true
              false
          
        
        
          
              ibiblio
              true
              false
              org/artifactory/**,org/jfrog/**
              http://repo1.maven.org/maven2
          
        
     
    Start Tomcat and navigate to http://localhost:8080/artifactory
    The artifactory home page is shown below:
    Sign in using username ‘admin’ and password ‘password’. Click on the Browse repository link and you should be able to view the contents of the repository.

    7. Configuring maven to use the new repository

    7.1 Configure maven using settings.xml
    Maven uses the settings.xml file located at ‘~/.m2/settings.xml’ to get the location of maven repository. If no repository is specified, maven uses the default repository which is at ibiblio.org. The settings.xml file has to be changed to use the new repository. The settings are shown below:
    
          
                dev
                
                      C:/InstalledPrograms/apache-tomcat-5.5.20
                
                
                      
                            central
                            http://localhost:8080/artifactory/repo
                            
                                  false
                            
                      
                      
                            snapshots
                            http://localhost:8080/artifactory/repo
                            
                                  false
                            
                      
                
                
                      
                            central
                            http://localhost:8080/artifactory/repo
                            
                                  false
                            
                      
                      
                            snapshots
                            http://localhost:8080/artifactory/repo
                            
                                  false
                            
                      
                
          
    
    7.2 Configure maven using project ‘pom.xml’
    The repository settings can also be done in the project pom.xml. A simple ‘pom.xml’ is shown below:
    
    4.0.0
    test
    test
    jar
    1.0-SNAPSHOT
    test
    http://maven.apache.org
           
                          
                                  central
                          http://localhost:8080/artifactory/repo
                          
                                  false
                          
                  
                  
                          snapshots
                          http://localhost:8080/artifactory/repo
                          
                                  false
                          
                  
          
          
                  
                          central
                          http://localhost:8080/artifactory/repo
                          
                                  false
                          
                  
                  
                          snapshots
                          http://localhost:8080/artifactory/repo
                          
                                  false
                          
                  
          
    
       
         junit
         junit
         3.8.1
         test
       
    
    
    7.3 Building using the new maven repository
    When building the maven project, all the repositories should be downloaded using the new repository. The console will show the server maven uses as shown below:
    Login to the new repository using your web browser and you will see that artifactory has downloaded and cached the artifacts from ibiblio.
    7.4 Installing artefacts to the repository
    Artifacts can be installed using the web UI or using maven command line. Installation using the web UI is simple and faster and does not require any configuration changes. Installation using the command line requires configuration changes in settings.xml and the it can be used in other scripts.
    7.4.1 Installing artifacts using the web UI
    The steps involved are shown below:
    1. Upload the artifact to deploy (‘jar’ file or ‘pom’ file)
    1. Artifactory will create the pom file if you are uploading a jar file. You can also specify which repository to upload to.
    1. Once uploaded, the artifact appears in the repository along with the ‘pom’ file created by artifactory.
    7.4.1 Installing artifacts from maven command line
    When using ‘mvn clean install’ command, maven only packages and installs the artifact to the local repository. To install it to the AHP internal repository, we have to add an additional configuration section in the settings.xml. The steps involved are shown below:
    
          
                
                      organisation-internal
                      admin
                      password
                
          
    
    To install an artefact to internal maven repository, the command is:
    mvn deploy:deploy-file -DrepositoryId=organisation-internal -Durl=http://localhost:8080/artifactory/private-internal-repository
    -DgroupId=test -DartifactId=test -Dversion=1.1 -Dpackaging=jar -Dfile=target/test-1.1.jar
    The repository id should match the server id defined in the settings.xml. The url should include the name of the repository the artefact is to be installed in.
    The new artifact appears in the repository and artifactory has created the ‘pom’ file for us automatically.

    8. Other features of Artifactory

    8.1 Backup the repository
    Backup policy is specified in the _FOLDER>/etc/artifactory.config.xml. Backup schedule is specified using ‘cron’ expression. The backup configuration element is highlighted below:
    
        
        0 0 /12 * * ?
        
          
              private-internal-repository
              Private internal repository
              true
              true
          
          
              3rd-party
              3rd party jars added manually
              true
              false
          
        
        
          
              ibiblio
              true
              false
              org/artifactory/**,org/jfrog/**
              http://repo1.maven.org/maven2
          
        
    
    Backups are stored in ‘_FOLDER>/backups’. The backups are in the same format as the local repository on developers machine. This makes it very easy to migrate the repository contents to another implementation of maven repository.
    8.2 Other features
    • Delete artifacts using the web UI
    • Search for artifacts using the web UI
    • Bulk import/export all artifacts in repository
    • The included jetty web server can be used if tomcat is not required

    9. Conclusion

    An internal private maven repository speeds up the build process and makes it easier to do clean builds. It also helps to avoid conflicts due to different versions of libraries.
    Amongst the 4 common maven repositories available, Artifactory seems to be the better product.
    Artifactory makes it easy to setup a maven repository. It provides all the features which a good maven repository should implement. The organization will not be locked into this tool as it is easy to migrate the repository contents to another implementation. A web UI makes the repository easy to use even for people who don’t know how the repository works.

    Biography

    Avneet Mangat has 5 years experience in Java/J2EE, and is currently working as Lead developer at Active Health Partners (www.ahp.co.uk).  He has a Bachelor’s degree in Software Engineering, and is a Sun Certified Java programmer, Adobe certified Flash Designer and Prince2 certified (foundation).  He is the lead developer of open source tool DBBrowser, please see http://databasebrowser.sourceforge.net/ .  His outside interests include photography and traveling, and he can be contacted at avneet.mangat@ahp.co.uk oravneet.mangat@gmail.com

    8 comments:

    oakleyses said...

    louis vuitton handbags, oakley sunglasses, louboutin, longchamp outlet, nike shoes, louis vuitton outlet stores, chanel handbags, burberry outlet, prada outlet, jordan shoes, tiffany and co, michael kors outlet, tory burch outlet, louis vuitton outlet, longchamp handbags, nike free, true religion jeans, michael kors outlet, kate spade outlet, polo ralph lauren outlet, tiffany and co, prada handbags, polo ralph lauren outlet, michael kors outlet, michael kors outlet, longchamp handbags, oakley sunglasses, ray ban sunglasses, kate spade handbags, burberry outlet, louis vuitton outlet, louboutin outlet, louboutin, coach factory outlet, air max, air max, coach outlet, gucci outlet, christian louboutin shoes, michael kors outlet, coach purses, ray ban sunglasses, michael kors outlet, louis vuitton, coach outlet store online, true religion jeans, oakley sunglasses cheap

    oakleyses said...

    ralph lauren, lululemon, air max, hollister, north face, nike air max, polo lacoste, vanessa bruno, timberland, vans pas cher, louboutin, louis vuitton, oakley pas cher, air max pas cher, nike roshe run, air max, true religion outlet, barbour, sac longchamp, air force, hollister, sac louis vuitton, nike free, polo ralph lauren, nike trainers, louis vuitton uk, nike roshe, sac hermes, longchamp, michael kors, sac burberry, sac guess, mulberry, new balance pas cher, converse pas cher, sac louis vuitton, hogan outlet, nike tn, north face, true religion outlet, ray ban pas cher, michael kors, air jordan, nike blazer, nike free pas cher, michael kors pas cher, abercrombie and fitch, ray ban sunglasses

    oakleyses said...

    mac cosmetics, mont blanc, marc jacobs, canada goose outlet, nike huarache, vans shoes, soccer jerseys, hollister, giuseppe zanotti, beats by dre, abercrombie and fitch, longchamp, insanity workout, celine handbags, bottega veneta, ghd, nfl jerseys, north face outlet, chi flat iron, ugg boots, birkin bag, ugg australia, canada goose, herve leger, ugg pas cher, rolex watches, valentino shoes, canada goose uk, canada goose, ferragamo shoes, canada goose, ugg boots, uggs outlet, north face jackets, soccer shoes, asics running shoes, new balance shoes, p90x, lululemon outlet, canada goose jackets, mcm handbags, instyler, babyliss pro, ugg, wedding dresses, jimmy choo outlet, reebok outlet, nike roshe run

    oakleyses said...

    parajumpers, karen millen, air max, converse, pandora charms, moncler, louboutin, moncler, links of london, lancel, juicy couture outlet, oakley, hollister, pandora charms, supra shoes, thomas sabo, canada goose, gucci, wedding dresses, timberland boots, swarovski crystal, air max, coach outlet store online, moncler, ray ban, canada goose, moncler, ugg, louis vuitton, swarovski, hollister, montre homme, moncler, hollister clothing store, ralph lauren, rolex watches, moncler outlet, moncler, iphone 6 cases, baseball bats, juicy couture outlet, toms shoes, vans, pandora jewelry, ugg, converse shoes

    oakleyses said...

    jordan pas cher, chanel handbags, nike outlet, michael kors pas cher, kate spade outlet, replica watches, longchamp pas cher, nike free, jordan shoes, christian louboutin shoes, nike free run, louis vuitton outlet, oakley sunglasses, ray ban sunglasses, polo ralph lauren, ugg boots, christian louboutin uk, air max, louis vuitton outlet, ugg boots, tiffany jewelry, polo ralph lauren outlet online, burberry pas cher, prada outlet, nike air max, gucci handbags, sac longchamp pas cher, ray ban sunglasses, louboutin pas cher, louis vuitton, uggs on sale, tiffany and co, oakley sunglasses, louis vuitton outlet, ray ban sunglasses, longchamp outlet, louis vuitton, longchamp outlet, replica watches, nike roshe, polo outlet, oakley sunglasses, cheap oakley sunglasses, oakley sunglasses wholesale, christian louboutin, christian louboutin outlet, tory burch outlet

    oakleyses said...

    lululemon canada, nike air max, burberry outlet, oakley pas cher, burberry handbags, coach outlet store online, kate spade, michael kors outlet, michael kors, nike air force, true religion jeans, true religion outlet, michael kors, polo lacoste, nike tn, new balance, abercrombie and fitch uk, michael kors outlet, uggs outlet, michael kors outlet, ralph lauren uk, michael kors outlet online, replica handbags, coach outlet, true religion outlet, coach purses, nike free uk, sac vanessa bruno, mulberry uk, michael kors outlet online, michael kors outlet online, michael kors outlet online, north face, uggs outlet, converse pas cher, hogan outlet, nike air max uk, hollister pas cher, sac hermes, nike roshe run uk, hollister uk, nike air max uk, true religion outlet, timberland pas cher, vans pas cher, ray ban pas cher, guess pas cher, ray ban uk

    oakleyses said...

    ipad cases, beats by dre, iphone 6 cases, ferragamo shoes, wedding dresses, hollister clothing, p90x workout, instyler, asics running shoes, nike huaraches, hermes belt, north face outlet, s6 case, babyliss, nfl jerseys, hollister, iphone 6 plus cases, insanity workout, ralph lauren, iphone cases, iphone 6s plus cases, bottega veneta, new balance shoes, longchamp uk, iphone 6s cases, giuseppe zanotti outlet, nike roshe run, mont blanc pens, herve leger, oakley, lululemon, timberland boots, nike air max, iphone 5s cases, ghd hair, valentino shoes, louboutin, nike trainers uk, celine handbags, north face outlet, baseball bats, vans outlet, abercrombie and fitch, chi flat iron, soccer jerseys, mac cosmetics, mcm handbags, soccer shoes, jimmy choo outlet, reebok outlet

    oakleyses said...

    supra shoes, marc jacobs, canada goose, converse outlet, toms shoes, ugg uk, ugg pas cher, canada goose outlet, canada goose, nike air max, louis vuitton, karen millen uk, doudoune moncler, converse, louis vuitton, vans, gucci, montre pas cher, moncler outlet, canada goose uk, canada goose outlet, moncler, pandora jewelry, barbour uk, moncler outlet, juicy couture outlet, louis vuitton, canada goose, pandora uk, coach outlet, swarovski crystal, ugg,uggs,uggs canada, juicy couture outlet, links of london, louis vuitton, louis vuitton, hollister, swarovski, canada goose outlet, ray ban, moncler, ugg, barbour, ugg,ugg australia,ugg italia, replica watches, pandora charms, moncler uk, pandora jewelry, wedding dresses, canada goose jackets, hollister, lancel