recentpopularlog in

kme : awk   57

linux - bash + arithmetic calculation with bash - Unix & Linux Stack Exchange
Since arithmetic expansion only does integer division, the proposed answer uses AWK, passing in values as variables, with '/dev/null' as the input file, and the math in a 'BEGIN' block.
bash  awk  shellscripting  floatingpoint  math  tipsandtricks  solution 
10 weeks ago by kme
How to view all the content in an awk array? - Unix & Linux Stack Exchange
You can loop over the array's keys and extract the corresponding values:
<code class="language-bash">awk '{my_dict[$1] = $2} END { for (key in my_dict) { print my_dict[key] } }' zen</code>

To get output similar to that you'd get with a Python dictionary, you can print the key as well:
<code class="language-bash">awk '{my_dict[$1] = $2} END { for (key in my_dict) { print key ": " my_dict[key] } }' zen

This works regardless of the key type.
awk  array  syntax  associativearray  solution  dammitbrain 
may 2019 by kme
bash - How to add a carriage return with sed? - Stack Overflow |
In this case, this guy was right.
sed is for simple subsitutions on individual lines, that is all. For anything else you should be using awk:
shellscripting  sed  awk  textprocessing  whitespace  sortof  solution 
march 2019 by kme
Genomics Code: How to correct negative coordinates from peaks bed file |
Use cat and pipe it to awk:
<code class="language-bash">cat peaks.bed | awk '{OFS="\t"}{if($2<0){$2=0};print}' > corrected_peaks.bed

Or remove them:
<code class="language-bash">cat peaks.bed | awk ‘($2>0)’ | filtered_peaks.bed</code>
bedfiles  bed  genomics  awk  tipsandtricks 
february 2019 by kme
In AWK, is it possible to specify "ranges" of fields? - Stack Overflow |
The answer is "no"; AWK does not support ranges of subscripts like Perl or Python slices.
unix  shellscripting  awk  syntax  newbie  workaround  solution 
january 2019 by kme
loop over characters in input string using awk - Stack Overflow |
You can convert a string to an array using split:
<code class="language-bash">
echo "here is a string" | awk '
split($0, chars, "")
for (i=1; i <= length($0); i++) {
printf("%s\n", chars[i])

This prints the characters vertically, one per line.
awk  arrays  textprocessing  shellscripting  solution 
november 2018 by kme
shell - Invoking a script, which has an awk shebang, with parameters (vars) - Stack Overflow |
Using 'env -S' actually works for jq, too (in OS X, anyway).
env is the easiest way to handle this problem:

#!/usr/bin/env -S awk -f

to add more options, and to ensure no interference with your arguments, and awk's arguments:

#!/usr/bin/env -S awk -F: -f ${_} --
# delete argv[1], which == ENVIRON[_]
delete ARGV[1]
} # rest of my awk program

as env has a POSIX standard, this shbang should get you around the difficulties of non-standard shbang implementations across unixen.


after having written this I realized that '-S' is a non-POSIX compliant FreeBSD env extension. So shell wrapper is probably the way to go, unfortunate as that is.
jq  awk  shellscripting  shebang  solution 
october 2017 by kme
linux - Using awk to print all columns from the nth to the last - Stack Overflow
will print all but very first column:

awk '{$1=""; print $0}' somefile

will print all but two first columns:

awk '{$1=$2=""; print $0}' somefile

@raphinesse you can fix that with awk '{$1=""; print substr($0,2)}' input_filename > output_filename – themiurgo Sep 12 '13 at 15:07
awk  shellscripting  solution 
august 2017 by kme
PyCressInt - Weirauch Lab TF Tools
will print all but very first column:

awk '{$1=""; print $0}' somefile

will print all but two first columns:

awk '{$1=$2=""; print $0}' somefile

@raphinesse you can fix that with awk '{$1=""; print substr($0,2)}' input_filename > output_filename – themiurgo Sep 12 '13 at 15:07
awk  shellscripting  solution 
august 2017 by kme
scripting - Best way to simulate "group by" from bash? - Stack Overflow
sort | uniq -c

Also very useful:


awk 'BEGIN { FS=OFS=SUBSEP="|"}{arr[$1,$2]+=$3+$4 }END {for (i in arr) print i,arr[i]}' file


Sort was used to handle very large volumes of data (think the original AT&T's billing data) on machines with 128k (that's 131,072 bytes) of memory (PDP-11). When sort encounters more data than a preset limit (often tuned close to the size of the machine's main memory) it sorts the data it has read in main memory and writes it into a temporary file. It then repeats the action with the next chunks of data. Finally, it performs a merge sort on those intermediate files. This allows sort to work on data many times larger than the machine's main memory.
bash  awk  sort  textprocessing  solution 
january 2017 by kme
sed - Merge two lines into one - Stack Overflow
Note the options using 'tr' and 'paste':
paste -d " " - - < filename

Related (
ls -1 | tr -d '\n'
shellscripting  newlines  whitespace  sed  awk  textprocessing  dammitbrain  maybesolution 
april 2016 by kme
Shell command to sum integers, one per line? - Stack Overflow
awk '{s+=$1} END {print s}' mydatafile


Paste typically merges lines of multiple files, but it can also be used to convert individual lines of a file into a single line. The delimiter flag allows you to pass a x+x type equation to bc.

paste -s -d+ infile | bc
shellscripting  shell  math  bash  awk  solution 
april 2016 by kme
removing a space from a string by awk - Stack Overflow
gsub is your friend. The following command basically does a global substitution of a regular expression (a single space in this case), replacing it with an empty string, on the target $0 (the whole line).

pax> echo "steve john" | awk '{ gsub (" ", "", $0); print}'
awk  shellscripting  scripting  textprocessing  solution 
july 2015 by kme
The GNU Awk User’s Guide: Using Shell Variables
printf "Enter search pattern: "
read pattern
awk -v pat="$pattern" '$0 ~ pat { nmatches++ }
END { print nmatches, "found" }' /path/to/data
awk  shell  programming  shellscripting  variables  solution 
april 2015 by kme
awk to split on all characters in a string
s="blah" echo $s | awk 'BEGIN{FS="";OFS="\n"}{$1=$1}1'
awk  commandline  incantation  textprocessing 
february 2015 by kme
Awk - A Tutorial and Introduction - by Bruce Barnett
The conditions do not have to be regular expressions. Relational tests can also be used. The following prints all lines between 20 and 40:

(NR==20),(NR==40) {print}
awk  shellscripting  solution  textprocessing 
august 2014 by kme
python - how do i do a range regex in ruby like awk /start/,/stop/ - Stack Overflow
I want to do an AWK-style range regex like this:

awk ' /hoststatus/,/\}/' file

To do a sed-style line number pattern, use NR==<start>,NR==<end> { whatever }.
awk  solution  incantation  shellscripting  textprocessing 
august 2014 by kme
awk command to merge columns from two separate files into single file? []
> cat file1
one two three
one two three
one two three
one two three

> cat file2
four five six
four five six
four five six
four five six

> pr -m -t -s\ file1 file2 | gawk '{print $4,$5,$6,$1}'
four five six one
four five six one
four five six one
four five six one
pr  cat  awk  shellscripting  solution  unix 
july 2011 by kme
AWK: compare two files -
Solution to finding the contents of one file in another and printing the matching lines (using grep -F)
awk  grep  shellscripting  solution 
february 2010 by kme

Copy this bookmark:

to read