With Microsoft's embrace of all things open source since Satya took over the helm more and more strange things seem to be possible.
This week i wanted to install the azcopy utility on linux to enable rapid copying of files to azure storage so i went about doing it - the first step to make this possible is to install dotnet for linux (yes you did hear that right).
Now we have a really old version of SLES and it's a 3rd party managed service so getting this working with no root rights and really old builds of the kernel (2006 - no joke) wasn't going to be easy (indeed if at all possible). To have any chance i would have to manully install all the components from source (i'd previously had to do this for other programs so i had manually installed a newer version of gcc, python and some other stuff - this is possible if a little fiddly). It's far easier to just use yum/zypper etc to load stuff with no worries about dependencies etc - you can see why this was done when you try and do stuff manually.....
Anyway i digress - let me show you how i installed it including a crafty fix i had to do right at the end to get round the fact i had an ancient version of the cp program.....
So first up we get the install script from microsoft - thats located here https://dot.net/v1/dotnet-install.sh in fact the whole steps for the install are well documented here https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x
Once i have that script on my system i then run it like this
./dotnet-install.sh --channel 2.0 --install-dir /oracle/home/oracle/dotnet
Initially it just hangs and fails - this is because of course it needs internet access and by default i don't have it on these machines so i manually set a http and https proxy as follows
export http_proxy=http://USERNAME:PASSWORD@proxyip:proxyport
export https_proxy=http://USERNAME:PASSWORD@proxyip:proxyport
Now any http(s) traffic is routed via the proxy and my requests get out.
So lets run again - this time i get a few warnings but one fail error which is...
dotnet_install: Error: Unable to locate libunwind. Install libunwind to continue
Now this isnt installed and i can't get the linux team to do it (the whole process is just too painful) - so instead i install it from source and then refer to that the basic steps to do that are:
wget http://download.savannah.nongnu.org/releases/libunwind/libunwind-1.2.1.tar.gz
gunzip libunwind-1.2.1.tar.gz
tar -xvf libunwind-1.2.1.tar
cd libunwind-1.2.1
./configure --prefix=/oracle/home/oracle/libunwind
make
make install
Now thats installed into /oracle/home/oracle/libunwind
Now i just need to tell the process to look for it there when i run the install script - to do that is just
export LD_LIBRARY_PATH=/oracle/home/oracle/libunwind
Now we get past the previous error and we start the download and execute process but then i get this
dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/2.0.3/dotnet-sdk-2.0.3-linux-x64.tar.gz
dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/2.0.3/dotnet-sdk-2.0.3-linux-x64.tar.gz
cp: invalid option -- n
Try `cp --help' for more information.
right......
What is the -n option in cp? In my SLES version it doesnt exist (hence the error).
On a newer os the man page shows this
-n, --no-clobber
do not overwrite an existing file (overrides a previous -i option)
But i'm stuck now right? I can't change the version of cp as I'm not root and i can;t just somehow compile my own cp command as it's a kind of os built in and can't be individually compiled?
At this point i thought i was stuck - then i thought about it some more - how could i get it to execute a different version of cp? I had 3 ideas....
1) alias cp -n so that it just ran cp
2) create a shell function called cp that took the command input, removed the -n then executed the actual cp command without it
3) create a shell script called cp that did the same as 2
So i tried all 3 and the results were...
1) Alias didn't like having spaces in the command (and actually i dodn't know what the actual command being run was - it could of being cp -a -n or something then a straight alias wouldnt match anyway) - so this was ruled out
2) The shell function worked fine in a test session but didn't work in the child process that was running as the subshell doesnt seem to pick it up - i think newer os's and some linux version can make this work - but not the one i had - so this was out
3) last chance saloon - but this one worked - and here is what i did.
I created a small shell script called cp in /oracle/home/oracle/bin with this content
RICH=`echo $@ | sed -e "s/-n//g" `
/bin/cp $RICH
Now all this does is take all the arguments passed in (thats the $@ variable) passes the whole lot to sed to do a pattern substitution - in this case globally replace -n with nothing and store the results of that in a variable called RICH. The next line then runs the actual built in cp command passing in the arguments that RICH now contains.
To make this now work i need to set the directory containing this 'dummy' cp command first in my PATH so when the shell is looking for cp it excutes my version rather than the built in one. A simple case looks like this.
export PATH=/oracle/home/oracle/bin:$PATH
Now when i type
cp -a -b -c -n filenamesource filenamedestination
The shell finds the cp command that i just created first in the PATH and runs that - this reads the command input replaces the -n with nothing - the command string then looks like this
-a -b -c filenamesource filenamedestination # note the missing -n
This string is then passed to /bin/cp (note i use the full path to pick the real cp otherwise i end up in a feedback look as just using cp would call my version again)
Now when i run the install script i see this
dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/2.0.3/dotnet-sdk-2.0.3-linux-x64.tar.gz
dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/2.0.3/dotnet-sdk-2.0.3-linux-x64.tar.gz
dotnet-install: Adding to current process PATH: `/oracle/home/oracle/dotnet`. Note: This change will be visible only when sourcing script.
dotnet-install: Installation finished successfully.
In the words of Kriss Akabusi - BOOM! :-)
Sorry couldnt resist that after seeing him at the UKOUG this year......
I was well pleased at this point and then just had to move on to install azcopy - however this time i wasn't so lucky - the ancient kernel was preventing me installing the version of glibc i needed as pre-requisite and all my great strides were for nothing......
However i did manage to get the older az command working - this was actually possible though did require a huge amount of fiddling around to make work (in fact so much fiddling that i kind of lost track myself and didn't write all the steps down....).
az does what i want but does require a full azure login to work to transfer files, where azcopy just needs account names and secret keys and is easier to work with
Comments
Post a Comment