Thursday, October 19, 2023

Gradle 8.4: "Convention type has been deprecated"

I upgraded Gradle to version 8.4 and started getting the following types of messages:

The org.gradle.api.plugins.ApplicationPluginConvention type has been deprecated. This is scheduled to be removed in Gradle 9.0. Consult the upgrading guide for further information: https://docs.gradle.org/8.4/userguide/upgrading_version_8.html#application_convention_deprecation

The org.gradle.api.plugins.Convention type has been deprecated. This is scheduled to be removed in Gradle 9.0. Consult the upgrading guide for further information: https://docs.gradle.org/8.4/userguide/upgrading_version_8.html#deprecated_access_to_conventions 

What's going on is that my existing build.gradle files contained some configuration options used in a way that got deprecated. The changes I had to make were largely cosmetic. I just had to group some of the existing configuration options into their own blocks:

I. Pre 8.4:

sourceCompatibility = 1.17
targetCompatibility = 1.17

mainClassName = "run.Main"
applicationDefaultJvmArgs = ["-Xmx1g"]

II. Post 8.4:

java {
  sourceCompatibility = 1.17
  targetCompatibility = 1.17
}

application {
  mainClass.set("run.Main")
  applicationDefaultJvmArgs = ["-Xmx1g"]
}

Tuesday, October 17, 2023

Remove duplicate pictures

Recently I wanted to remove duplicate photos I had based on image content, not just file hash.
This program, AllDup by MTSD, did a great job.

https://www.alldup.de/en_download_alldup.php

Sunday, October 15, 2023

"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results."

I was playing with Huggingface transformers and kept getting the warning "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.". I finally found a solution in a StackOverflow reply that will be credited at the end:

To fix this, first add this code after loading pre-trained tokenizer:

if tokenizer.pad_token is None:
  tokenizer.pad_token = tokenizer.eos_token

Then pass this in generate method like this:

gen_ids = model.generate(**encodings, pad_token_id=tokenizer.pad_token_id, max_new_tokens=200)

In short, there are two additions/changes you need to make:
  1. When initializing your tokenizer, set:
    tokenizer.pad_token = tokenizer.eos_token
  2. When using the model to generate an output, pass the following as a parameter to model.generate:
    pad_token_id=tokenizer.pad_token_id
Thank you user Shital Shah on StackOverflow:

"A decoder-only architecture is being used, but right-padding was detected!"

I was playing with Huggingface transformers and kept getting the warning "A decoder-only architecture is being used, but right-padding was detected!". I finally found a solution in a StackOverflow reply that will be credited at the end:

Padding in this context is referring to the "tokenizer.eos_token", and you are currently padding to the right of the user input and the error is saying that for correct results add padding to the left. You need to do this:

new_user_input_ids = tokenizer.encode(tokenizer.eos_token + input(">> User:"), return_tensors='pt')

While I originally thought it was about setting the parameter padding_side='left', it turned out to be about the order in which you concatenate the input and the eos_token.

Thank you user Travis Thayer on StackOverflow:
https://stackoverflow.com/questions/74748116/huggingface-automodelforcasuallm-decoder-only-architecture-warning-even-after/74972288#74972288

Wednesday, October 11, 2023

Update apt packages

#!/usr/bin/env bash
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get autoremove
sudo apt-get clean 

Wednesday, October 4, 2023

Install specific Go version on Linux

A bash script for installing a specific Go version on Linux (AMD64).
Expects a numeric version as a parameter (e.g. 1.21.1).

Simplified:

architecture=$(dpkg --print-architecture)
file_name="go${version}.linux-${architecture}.tar.gz"

wget https://go.dev/dl/${file_name}"
rm -rf /usr/local/go && tar -C /usr/local -xzf ${file_name}

Write to ~/.profile

export PATH=$PATH:/usr/local/go/bin

Monday, October 2, 2023

Sub7

Interview with Sub7 creator, Mobman:
- https://twitter.com/DarkCoderSc/status/1681208015255379968
- https://darkcodersc.medium.com/a-malware-retrospective-subseven-d86fed0c88bf

Born and raised in Craiova, Romania, Mobman was drawn to the world of software and malware at an early age. His fascination led him to the creation of the infamous SubSeven Remote Access Trojan, a feat achieved under a pseudonym inspired by his enduring favorite band, B.U.G. Mafia. As he reflected, “The nickname was inspired from my favorite band (still to this day!), the Romanian rap group called B.U.G. Mafia. I wanted to pick something mob-related and mobman just had a nice ring to it.”.

Sub7 fun fact: mobman used to write feature ideas in notebooks
https://twitter.com/xillwillx/status/1708766696985575772