Package de.uni_mannheim.informatik.dws.melt.matching_ml.python.nlptransformers

Class TransformersFineTuner

java.lang.Object

eu.sealsproject.platform.res.tool.impl.AbstractPlugin

de.uni_mannheim.informatik.dws.melt.matching_base.MatcherURL

de.uni_mannheim.informatik.dws.melt.matching_base.MatcherFile

de.uni_mannheim.informatik.dws.melt.matching_jena.MatcherYAAA

de.uni_mannheim.informatik.dws.melt.matching_jena.MatcherYAAAJena

de.uni_mannheim.informatik.dws.melt.matching_ml.python.nlptransformers.TransformersBase

de.uni_mannheim.informatik.dws.melt.matching_ml.python.nlptransformers.TransformersBaseFineTuner

de.uni_mannheim.informatik.dws.melt.matching_ml.python.nlptransformers.TransformersFineTuner

All Implemented Interfaces:

Filter, IMatcher<org.apache.jena.ontology.OntModel,Alignment,Properties>, eu.sealsproject.platform.res.domain.omt.IOntologyMatchingToolBridge, eu.sealsproject.platform.res.tool.api.IPlugin, eu.sealsproject.platform.res.tool.api.IToolBridge

Direct Known Subclasses:

RelationTypeFineTuner, TransformersFineTunerHpSearch

public class TransformersFineTuner
extends TransformersBaseFineTuner
implements Filter

This class is used to fine-tune a transformer model based on a generated dataset. In every call to the match method, the training data will be generated and appended to a temporary file. When you call the TransformersBaseFineTuner.finetuneModel() method, then a model is fine-tuned and the training file is deleted.

Field Summary

Fields

Modifier and Type	Field	Description
protected BatchSizeOptimization	batchSizeOptimization
private static final org.slf4j.Logger	LOGGER

Fields inherited from class de.uni_mannheim.informatik.dws.melt.matching_ml.python.nlptransformers.TransformersBaseFineTuner

additionallySwitchSourceTarget, NEWLINE, resultingModelLocation, trainingFile

Fields inherited from class de.uni_mannheim.informatik.dws.melt.matching_ml.python.nlptransformers.TransformersBase

cudaVisibleDevices, extractor, modelName, multipleTextsToMultipleExamples, multiProcessing, trainingArguments, transformersCache, usingTensorflow

Fields inherited from class de.uni_mannheim.informatik.dws.melt.matching_base.MatcherFile

FILE_PREFIX, FILE_SUFFIX

Constructor Summary

Constructors

Constructor	Description
TransformersFineTuner(TextExtractorMap extractor, String initialModelName, File resultingModelLocation)	Run the training of a NLP transformer.
TransformersFineTuner(TextExtractor extractor, String initialModelName, File resultingModelLocation)	Run the training of a NLP transformer.

Method Summary

Modifier and Type	Method	Description
void	addTrainingParameterToMakeTrainingFaster()	This will add (potencially multiple) training parameters to the current trainingArguments to make the training faster.
File	finetuneModel(File trainingFile)	Finetune a given model with the provided text in the csv file (three columns: first text, second text, label(0/1))
BatchSizeOptimization	getBatchSizeOptimization()
int	getMaximumPerDeviceTrainBatchSize()	This functions tries to execute the training with one step to check which maximum per_device_train_batch_size is possible.It will start with 2 and checks only powers of 2.
int	getMaximumPerDeviceTrainBatchSize(File trainingFile)	This functions tries to execute the training with one step to check which maximum per_device_train_batch_size is possible.It will start with 2 and checks only powers of 2.
boolean	isAdjustMaxBatchSize()	Returns the value if max batch size is adjusted or not.
void	setAdjustMaxBatchSize(boolean adjustMaxBatchSize)	If set to true, then it will set the max value of the search space for the training batch size to the maximum which is possible with the current GPU/CPU.
void	setBatchSizeOptimization(BatchSizeOptimization batchSizeOptimization)

Methods inherited from class de.uni_mannheim.informatik.dws.melt.matching_ml.python.nlptransformers.TransformersBaseFineTuner

clearTrainingData, createTrainingFile, finetuneModel, getResultingModelLocation, getTrainingFile, isAdditionallySwitchSourceTarget, match, setAdditionallySwitchSourceTarget, setResultingModelLocation, writeClassificationFormat, writeTrainingFile

Methods inherited from class de.uni_mannheim.informatik.dws.melt.matching_ml.python.nlptransformers.TransformersBase

addTrainingArgument, getCudaVisibleDevices, getCudaVisibleDevicesButOnlyOneGPU, getExamplesForBatchSizeOptimization, getExtractor, getExtractorMap, getModelName, getMultiProcessing, getTextualRepresentation, getTrainingArguments, getTransformersCache, isMultipleTextsToMultipleExamples, isOptimizeForMixedPrecisionTraining, isUsingTensorflow, setCudaVisibleDevices, setCudaVisibleDevices, setExtractor, setExtractorMap, setModelName, setMultipleTextsToMultipleExamples, setMultiProcessing, setOptimizeForMixedPrecisionTraining, setTrainingArguments, setTransformersCache, setUsingTensorflow, writeExamplesToFile

Methods inherited from class de.uni_mannheim.informatik.dws.melt.matching_jena.MatcherYAAAJena

getModelSpec, match, readOntology

Methods inherited from class de.uni_mannheim.informatik.dws.melt.matching_jena.MatcherYAAA

match

Methods inherited from class de.uni_mannheim.informatik.dws.melt.matching_base.MatcherFile

match

Methods inherited from class de.uni_mannheim.informatik.dws.melt.matching_base.MatcherURL

align, align, canExecute, getType

Methods inherited from class eu.sealsproject.platform.res.tool.impl.AbstractPlugin

getId, getVersion, setId, setVersion

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface eu.sealsproject.platform.res.tool.api.IPlugin

getId, getVersion

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

batchSizeOptimization

protected BatchSizeOptimization batchSizeOptimization

Constructor Details

TransformersFineTuner

public TransformersFineTuner(TextExtractorMap extractor,
 String initialModelName,
 File resultingModelLocation)

Run the training of a NLP transformer.

Parameters:

extractor - used to extract text from a given resource. This is the text which represents a resource.

initialModelName - the initial model name for fine tuning which can be downloaded or a path to a directory containing model weights ( see first parameter pretrained_model_name_or_path of the from_pretrained function in huggingface library). This value can be also changed by TransformersBase.setModelName(java.lang.String).

resultingModelLocation - the final location where the fine-tuned model should be stored.

TransformersFineTuner

public TransformersFineTuner(TextExtractor extractor,
 String initialModelName,
 File resultingModelLocation)

Run the training of a NLP transformer.

Parameters:

extractor - used to extract text from a given resource. This is the text which represents a resource.

initialModelName - the initial model name for fine tuning which can be downloaded or a path to a directory containing model weights ( see first parameter pretrained_model_name_or_path of the from_pretrained function in huggingface library). This value can be also changed by TransformersBase.setModelName(java.lang.String).

resultingModelLocation - the final location where the fine-tuned model should be stored.

Method Details

finetuneModel

public File finetuneModel(File trainingFile)
                   throws Exception

Description copied from class: TransformersBaseFineTuner

Finetune a given model with the provided text in the csv file (three columns: first text, second text, label(0/1))

Specified by:

finetuneModel in class TransformersBaseFineTuner

Parameters:

trainingFile - csv file with three columns: first text, second text, label(0/1) (can be generated with TransformersBaseFineTuner.createTrainingFile(OntModel, OntModel, Alignment) )

Returns:

the final location (directory) of the finetuned model (which is also given in the constructor)

Throws:

Exception - in case of any error

getMaximumPerDeviceTrainBatchSize

public int getMaximumPerDeviceTrainBatchSize()

This functions tries to execute the training with one step to check which maximum per_device_train_batch_size is possible.It will start with 2 and checks only powers of 2. It uses the data collected by running this fine tuner on testcases. If you have a file (comma separated) then you can use getMaximumPerDeviceTrainBatchSize(java.io.File).

Returns:

the maximum per_device_train_batch_size with the current configuration

getMaximumPerDeviceTrainBatchSize

public int getMaximumPerDeviceTrainBatchSize(File trainingFile)

This functions tries to execute the training with one step to check which maximum per_device_train_batch_size is possible.It will start with 2 and checks only powers of 2. It uses the data collected by running this fine tuner on testcases.

Parameters:

trainingFile - the training file to use

Returns:

the maximum per_device_train_batch_size with the current configuration

addTrainingParameterToMakeTrainingFaster

public void addTrainingParameterToMakeTrainingFaster()

This will add (potencially multiple) training parameters to the current trainingArguments to make the training faster. Thus do not change the trainingArguments object afterwards (with setTrainingArguments ). What you can do is to add more training arguments via getTrainingArguments.addParameter (as long as you do not modify any parameters added by this method). The following parameters are set:

• fp16

See Also:

• Transformers performance page on huggingface

isAdjustMaxBatchSize

public boolean isAdjustMaxBatchSize()

Returns the value if max batch size is adjusted or not.

Returns:

true if the batch size is modified.

setAdjustMaxBatchSize

public void setAdjustMaxBatchSize(boolean adjustMaxBatchSize)

If set to true, then it will set the max value of the search space for the training batch size to the maximum which is possible with the current GPU/CPU.

Parameters:

adjustMaxBatchSize - true to enable the adjustment

getBatchSizeOptimization

public BatchSizeOptimization getBatchSizeOptimization()

setBatchSizeOptimization

public void setBatchSizeOptimization(BatchSizeOptimization batchSizeOptimization)