CodeBERTa-small-v1-SOCO-Java-SoftmaxLoss-2

SentenceTransformer based on huggingface/CodeBERTa-small-v1

This is a sentence-transformers model finetuned from huggingface/CodeBERTa-small-v1 on the soco_java dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: huggingface/CodeBERTa-small-v1 <!-- at revision e93b5898cff07f03f1c1c09cde284d1b85962363 -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • soco_java

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("buelfhood/CodeBERTa-small-v1-SOCO-Java-SoftmaxLoss-2")
# Run inference
sentences = [
    '\nimport java.util.*;\nimport java.io.*;\nimport java.net.*;\n\nclass BruteForce\n{\n\n public static void main (String a[])\n {\n \n final char [] alphabet = {\n        \'A\', \'B\', \'C\', \'D\', \'E\', \'F\', \'G\', \'H\',\n        \'I\', \'J\', \'K\', \'L\', \'M\', \'N\', \'O\', \'P\',\n        \'Q\', \'R\', \'S\', \'T\', \'U\', \'V\', \'W\', \'X\',\n        \'Y\', \'Z\', \'a\', \'b\', \'c\', \'d\', \'e\', \'f\',\n        \'g\', \'h\', \'i\', \'j\', \'k\', \'l\', \'m\', \'n\',\n        \'o\', \'p\', \'q\', \'r\', \'s\', \'t\', \'u\', \'v\',\n        \'w\', \'x\', \'y\', \'z\'};\n\n String pwd="";\n \n for(int i=0;i<52;i++)\n {\n  for(int j=0;j<52;j++)\n  {\n   for(int k=0;k<52;k++)\n   {\n    pwd = alphabet[i]+""+alphabet[j]+""+alphabet[k];\n    String userPassword = ":"+pwd;\n    RealThread myTh = new RealThread(i,userPassword);\n    Thread th = new Thread( myTh );\n    th.start();\n    try\n    {\n     \n     \n     th.sleep(100);\n    }\n    catch(Exception e)\n    {} \n   }\n  }\n }\n\n\n}\n\n\n}\n\n\nclass RealThread implements Runnable\n{\n private int num;\n private URL url;\n private HttpURLConnection uc =null;\n private String userPassword;\n private int responseCode = 100;\n public RealThread (int i, String userPassword)\n {\n try\n {\n url = new URL("http://sec-crack.cs.rmit.edu./SEC/2/");\n }\n catch(Exception ex1)\n {\n }\n num = i;\n this.userPassword = userPassword;\n\n }\n \n public int getResponseCode()\n {\n\n return this.responseCode;\n }\n\n public void run()\n {\n  try\n  {\n  String encoding = new url.misc.BASE64Encoder().encode (userPassword.getBytes());\n\n  uc = (HttpURLConnection)url.openConnection();\n  uc.setRequestProperty ("Authorization", " " + encoding);\n  System.out.println("Reponse  = "+uc.getResponseCode()+"for pwd = "+userPassword);\n  this.responseCode = uc.getResponseCode();\n  \n  if(uc.getResponseCode()==200)\n  {\n     System.out.println(" ======= Password Found : "+userPassword+" ========================================= ");\n     System.exit(0);\n  }\n\n  }\n  catch (Exception e) {\n  System.out.println("Could not execute Thread "+num+" ");\n  }\n }\n\n}\n',
    'import java.io.BufferedReader;\nimport java.io.FileInputStream;\nimport java.io.IOException;\nimport java.io.InputStreamReader;\nimport java.util.Date;\nimport java.util.Properties;\n\nimport javax.mail.Message;\nimport javax.mail.Session;\nimport javax.mail.Transport;\nimport javax.mail.Message.RecipientType;\nimport javax.mail.internet.InternetAddress;\nimport javax.mail.internet.MimeMessage;\n\n\n\n\npublic class Mailsend\n{\n    static final String SMTP_SERVER = MailsendPropertyHelper.getProperty("smtpServer");\n    static final String RECIPIENT_EMAIL = MailsendPropertyHelper.getProperty("recipient");\n    static final String SENDER_EMAIL = MailsendPropertyHelper.getProperty("sender");\n    static final String MESSAGE_HEADER = MailsendPropertyHelper.getProperty("messageHeader");\n\n\n\t\n\n\tpublic static void main(String args[])\n\t{\n\t\ttry\n\t\t{\n\t\t\t\n\t\t\tString smtpServer = SMTP_SERVER;\n\t\t\tString recip = RECIPIENT_EMAIL;\n\t\t\tString from = SENDER_EMAIL;\n\t\t\tString subject = MESSAGE_HEADER;\n\t\t\tString body = "Testing";\n\n\t\t\tSystem.out.println("Started sending the message");\n\t\t\tMailsend.send(smtpServer,recip , from, subject, body);\n\t\t}\n\t\tcatch (Exception ex)\n\t\t{\n\t\t\tSystem.out.println(\n\t\t\t\t"Usage: java mailsend"\n\t\t\t\t\t+ " smtpServer toAddress fromAddress subjectText bodyText");\n\t\t}\n\n\t\tSystem.exit(0);\n\t}\n\n\n\t\n\tpublic static void send(String smtpServer, String receiver,\tString from, String subject, String body)\n\n\t{\n\t\ttry\n\t\t{\n\t\t\tProperties props = System.getProperties();\n\n\t\t\t\n\n\t\t\tprops.put("mail.smtp.host", smtpServer);\n\t\t\tprops.put("mail.smtp.timeout", "20000");\n\t\t\tprops.put("mail.smtp.connectiontimeout", "20000");\n\n\t\t\t\n\t\t\tSession session = Session.getDefaultInstance(props, null);\n\n\n\t\t\t\n\t\t\tMessage msg = new MimeMessage(session);\n\n\t\t\t\n\t\t\tmsg.setFrom(new InternetAddress(from));\n\t\t\tmsg.setRecipients(Message.RecipientType.NORMAL,\tInternetAddress.parse(receiver, false));\n\n\n\n\t\t\t\n\t\t\tmsg.setSubject(subject);\n\n\t\t\tmsg.setSentDate(new Date());\n\n\t\t\tmsg.setText(body);\n\n\t\t\t\n\t\t\tTransport.send(msg);\n\n\t\t\tSystem.out.println("sent the email with the differences : "+ + "using the mail server: "+ smtpServer);\n\n\t\t}\n\t\tcatch (Exception ex)\n\t\t{\n\t\t\tex.printStackTrace();\n\t\t}\n\t}\n}\n',
    '\n\n\n\n\n\nimport java.util.*;\nimport java.io.*;\nimport java.net.*;\n\npublic class Watchdog extends TimerTask\n{\n\tpublic void run()\n\t{\n\t\tRuntime t = Runtime.getRuntime();\n\t  \tProcess pr= null;\n\t  \tString Fmd5,Smd5,temp1;\n\t  \tint index;\n          \n\t \ttry\n          \t{\n\t\t    \n\t\t    pr = t.exec("md5sum csfirst.html");\n\n                    InputStreamReader stre = new InputStreamReader(pr.getInputStream());\n                    BufferedReader bread = new BufferedReader(stre);\n\t\t    \n\t\t    s = bread.readLine();\n\t\t    index = s.indexOf(\' \');\n\t\t    Fmd5 = s.substring(0,index);\n\t\t    System.out.println(Fmd5);\n\t\t    \n\t\t    pr = null;\n\t\t    \n\t\t    pr = t.exec("wget http://www.cs.rmit.edu./students/");\n\t\t    pr = null;\n\t\t    \n\t\t    pr = t.exec("md5sum index.html");\n\t\t    \n\n\t\t    InputStreamReader stre1 = new InputStreamReader(pr.getInputStream());\n                    BufferedReader bread1 = new BufferedReader(stre1);\n\t\t    \n\t\t    temp1 = bread1.readLine();\n\t\t    index = temp1.indexOf(\' \');\n\t\t    Smd5 = temp1.substring(0,index);\n\t\t    System.out.println(Smd5);\n\t\t\n\t\t    pr = null;\n\t\t\n\t\t    if(Fmd5 == Smd5)\n\t\t       System.out.println("  changes Detected");\n\t\t    else\n\t\t    {\n\t\t       pr = t.exec("diff csfirst.html index.html > report.html");\n\t\t       pr = null;\n\t\t       \n\t\t       try{\n\t\t       Thread.sleep(10000);\n\t\t       }catch(Exception e){}\n\t\t       \n\t\t       pr = t.exec(" Message.txt | mutt -s Chnages  Webpage -a report.html -x @yallara.cs.rmit.edu.");\n\t\t     \n\t\t       \n\t\t       \n\t\t    }   \n\t\t    \n    \t        }catch(java.io.IOException e){}\n\t}\n}\t\t\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

soco_java

• Dataset: soco_java
• Size: 30,069 training samples
• Columns: <code>label</code>, <code>text_1</code>, and <code>text_2</code>
• Approximate statistics based on the first 1000 samples:

  	label	text_1	text_2
  type	int	string	string
  details	<ul><li>0: ~99.70%</li><li>1: ~0.30%</li></ul>	<ul><li>min: 51 tokens</li><li>mean: 450.65 tokens</li><li>max: 512 tokens</li></ul>	<ul><li>min: 51 tokens</li><li>mean: 468.5 tokens</li><li>max: 512 tokens</li></ul>

• Samples:

  label	text_1	text_2
  <code>0</code>	<code><br><br><br><br> <br>import java.io.;<br>import java.net.;<br>import java.Runtime;<br>import java.util.*;<br>import java.net.smtp.SmtpClient; <br><br><br><br>public class WatchDog<br><br>{<br><br> static String strImageOutputFile01 = "WebPageImages01.txt";<br> static String strImageOutputFile02 = "WebPageImages02.txt";<br><br> static String strWebPageOutputFile01 = "WebPageOutput01.txt";<br> static String strWebPageOutputFile02 = "WebPageOutput02.txt";<br><br> static String strWatchDogDiffFile_01_02 = "WatchDogDiff_01_02.txt";<br><br> static String strFromEmailDefault = "@.rmit.edu.";<br> static String strToEmailDefault = "@.rmit.edu.";<br><br> static String strFromEmail = null;<br> static String strToEmail = null;<br><br><br><br><br> public static void main (String args[])<br> <br> {<br><br> <br> <br> <br> <br> <br><br> URL url = null;<br> HttpURLConnection urlConnection;<br> int intContentLength;<br> String strWebPageText = "";<br><br> String strURL = "http://www.cs.rmit.edu./students/";<br> String strPrePend = "...</code>	<code>import java.io.;<br>import java.net.;<br>import java.util.*;<br><br>public class Watchdog<br>{<br> public static void main(String args[])<br> {<br> <br> String mainLink="http://www.cs.rmit.edu./students/";<br> String sender = "@cs.rmit.edu.";<br> String recipient = "<webtech@acuneeds.>";<br> String hostName = "yallara.cs.rmit.edu.";<br> int delay = 86400000;<br><br> try<br> {<br> int imgSrcIndex, imgSrcEnd;<br> String imgLink;<br> Vector imageList = new Vector();<br> HttpURLConnection imgConnection;<br> URL imgURL;<br><br> <br> EmailClient email = new EmailClient(sender, recipient, hostName);<br><br> <br> URL url=new URL(mainLink);<br> HttpURLConnection connection = (HttpURLConnection) url.openConnection();<br><br> BufferedReader webpage = new BufferedReader(new InputStreamReader(connection.getInputStream()));<br><br> <br> FileWriter fwrite = new FileWriter("local.txt");<br> BufferedWriter writefile = new BufferedWriter(fwrite);<br><br> String line=webpage.readLine();<br><br> while (line != null)<br> {<br> <br> writefile.write(line,0,line.length());<br> wri...</code>
  <code>0</code>	<code>import java.util.;<br>import java.io.;<br>import java.;<br><br>public class Dogs5<br>{<br> public static void main(String [] args) throws Exception<br> { <br> executes("rm index.");<br> executes("wget http://www.cs.rmit.edu./students");<br><br> while (true)<br> {<br> String addr= "wget http://www.cs.rmit.edu./students";<br> executes(addr);<br> String hash1 = md5sum("index.html");<br> String hash2 = md5sum("index.html.1");<br> System.out.println(hash1 +"	"+ hash2);<br> <br> BufferedReader buf = new BufferedReader(new FileReader("/home/k//Assign2/ulist1.txt"));<br><br> String line=" " ;<br> String line1=" " ;<br> String line2=" ";<br> String line3=" ";<br> String[] cad = new String[10];<br> <br> executes("./.sh");<br> <br> int i=0;<br> while ((line = buf.readLine()) != null)<br> {<br> <br> line1="http://www.cs.rmit.edu./students/images"+line;<br> if (i==1)<br> line2="http://www.cs.rmi...</code>
  <code>0</code>	<code><br><br>import java.util.;<br>import java.text.;<br>import java.io.;<br>import java.;<br>import java.net.*;<br><br>public class WatchDog<br>{<br> public static void main(String args[])<br> {<br> String s = null;<br> String webpage = "http://www.cs.rmit.edu./students/";<br> <br> <br> String file1 = "file1";<br> String file2 = "file2";<br> <br> try<br> {<br> Process p = Runtime.getRuntime().exec("wget -O " + file1 + " " + webpage);<br> <br> BufferedReader stdInput = new BufferedReader(new <br> InputStreamReader(p.getInputStream()));<br><br> BufferedReader stdError = new BufferedReader(new <br> InputStreamReader(p.getErrorStream()));<br><br> <br> while ((s = stdInput.readLine()) != null) { <br> System.out.println(s);<br> }<br> <br> <br> while ((s = stdError.readLine()) != null) { <br> System.out.println(s);<br> }<br> <br> try<br> {<br> p.waitFor(); <br> }<br> catch...</code>	<code><br><br>import java.io.;<br>import java.net.;<br>import java.util.;<br>import java.String;<br>import java.Object;<br>import java.awt.;<br><br><br><br>public class WatchDog<br>{<br> private URL url;<br> private URLConnection urlcon;<br> private int lastModifiedSince = 0;<br> private int lastModified[] = new int[2];<br><br> private int count = 0;<br><br> public static String oldFile;<br> public static String newFile;<br><br> private String diffFile;<br><br> private BufferedWriter bw;<br> private Process p;<br> private Runtime r;<br> private String fileName;<br><br> <br> <br> private ArrayList old[]= new ArrayList[500];<br> private ArrayList news[] = new ArrayList[500];<br> private String info = "";<br> private int index = 0;<br><br> public WatchDog(String fileName)<br> {<br> this.fileName = fileName;<br> oldFile = fileName + ".old";<br> newFile = fileName + ".new";<br> diffFile = "testFile.txt";<br> }<br> public static void main(String args[])<br> {<br> WatchDog wd = new WatchDog("TestDog");<br><br> wd.detectChange(WatchDog.oldFile);<br> while (true)<br> {<br> try<br> {<br> Thread.slee...</code>

• Loss: <code>SoftmaxLoss</code>

Evaluation Dataset

soco_java

• Dataset: soco_java at c8fab14
• Size: 3,342 evaluation samples
• Columns: <code>label</code>, <code>text_1</code>, and <code>text_2</code>
• Approximate statistics based on the first 1000 samples:

  	label	text_1	text_2
  type	int	string	string
  details	<ul><li>0: ~99.40%</li><li>1: ~0.60%</li></ul>	<ul><li>min: 51 tokens</li><li>mean: 443.11 tokens</li><li>max: 512 tokens</li></ul>	<ul><li>min: 51 tokens</li><li>mean: 467.05 tokens</li><li>max: 512 tokens</li></ul>

• Samples:

  label	text_1	text_2
  <code>0</code>	<code><br><br>import java.Runtime;<br>import java.io.*;<br><br>public class differenceFile<br>{<br> StringWriter sw =null;<br> PrintWriter pw = null;<br> public differenceFile()<br> {<br> sw = new StringWriter();<br> pw = new PrintWriter();<br> }<br> public String compareFile()<br> {<br> try<br> {<br> Process = Runtime.getRuntime().exec("diff History.txt Comparison.txt");<br><br> InputStream write = sw.getInputStream();<br> BufferedReader bf = new BufferedReader (new InputStreamReader(write));<br> String line;<br> while((line = bf.readLine())!=null)<br> pw.println(line);<br> if((sw.toString().trim()).equals(""))<br> {<br> System.out.println(" difference");<br> return null;<br> }<br> System.out.println(sw.toString().trim());<br> }catch(Exception e){}<br> return sw.toString().trim();<br> }<br>}</code>	<code><br><br><br><br><br><br><br>import java.;<br>import java.io.;<br>import java.util.*;<br><br>public class BruteForce<br>{<br><br> public static void main(String[] args) <br> {<br> Runtime rt = Runtime.getRuntime();<br> Process pr= null;<br> char chars[] = {'a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z','A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'};<br> String pass;<br> char temp[] = {'a','a'};<br> char temp1[] = {'a','a','a'};<br> char temp2[] = {'a'};<br><br> String f= new String();<br> String resp = new String();<br> int count=0;<br> String success = new String();<br> InputStreamReader instre;<br> BufferedReader bufread;<br><br><br> for(int k=0;k<52;k++)<br> {<br> temp2[0]=chars[k];<br> pass = new String(temp2); <br> count++; <br><br> System.out.println("The password tried ...</code>
  <code>0</code>	<code>import java.io.;<br>import java.net.;<br>import java.util.*;<br><br>public class Watchdog<br>{<br> public static void main(String args[])<br> {<br> <br> String mainLink="http://www.cs.rmit.edu./students/";<br> String sender = "@cs.rmit.edu.";<br> String recipient = "<webtech@acuneeds.>";<br> String hostName = "yallara.cs.rmit.edu.";<br> int delay = 86400000;<br><br> try<br> {<br> int imgSrcIndex, imgSrcEnd;<br> String imgLink;<br> Vector imageList = new Vector();<br> HttpURLConnection imgConnection;<br> URL imgURL;<br><br> <br> EmailClient email = new EmailClient(sender, recipient, hostName);<br><br> <br> URL url=new URL(mainLink);<br> HttpURLConnection connection = (HttpURLConnection) url.openConnection();<br><br> BufferedReader webpage = new BufferedReader(new InputStreamReader(connection.getInputStream()));<br><br> <br> FileWriter fwrite = new FileWriter("local.txt");<br> BufferedWriter writefile = new BufferedWriter(fwrite);<br><br> String line=webpage.readLine();<br><br> while (line != null)<br> {<br> <br> writefile.write(line,0,line.length());<br> wri...</code>	<code><br><br>import java.net.;<br>import java.io.;<br>import java.String;<br>import java.;<br>import java.util.;<br><br>public class BruteForce {<br> private static final int passwdLength = 3; <br> private static String commandLine<br> = "curl http://sec-crack.cs.rmit.edu./SEC/2/index.php -I -u :";<br> private String chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";<br> private int charLen = chars.length(); <br> private int n = 0; <br> private int n3 = charLencharLencharLen; <br> private String response;<br> private String[] password = new String[charLencharLencharLen+charLen*charLen+charLen];<br> private char[][] data = new char[passwdLength][charLen];<br> private char[] pwdChar2 = new char[2];<br> private char[] pwdChar = new char[passwdLength];<br> private String url;<br> private int startTime;<br> private int endTime;<br> private int totalTime;<br> private float averageTime;<br> private boolean finish;<br> private Process curl;<br> private BufferedReader bf, responseLine;<br><br>...</code>
  <code>0</code>	<code><br>import java.io.;<br>import java.awt.;<br>import java.net.*;<br><br>public class BruteForce<br>{<br> public static void main (String[] args)<br> {<br> String pw = new String();<br> pw = getPassword ();<br> System.out.println("Password is: "+pw);<br> }<br> public static String getPassword()<br> {<br> String passWord = new String();<br> passWord = "AAA";<br> char[] guess = passWord.toCharArray();<br> Process pro = null;<br> Runtime runtime = Runtime.getRuntime();<br> BufferedReader in = null;<br> String str=null;<br> boolean found = true;<br><br> System.out.println(" attacking.....");<br> for (int i=65;i<=122 ;i++ )<br> {<br> guess[0]=(char)(i);<br> for (int j=65;j<=122 ;j++ )<br> {<br> guess[1]=(char)(j);<br> for (int k=65 ;k<=122 ;k++ )<br> {<br> guess[2]=(char)(k);<br> passWord = new String(guess);<br> String cmd = "wget --http-user= --http-passwd="+passWord +" http://sec-crack.cs.rmit.edu./SEC/2/index.php ";<br> try<br> {<br> pro = runtime.exec(cmd);<br><br> in = new BufferedReader(new InputStreamReader(pro.getErrorSt...</code>	<code><br><br>import java.io.;<br>import java.text.;<br>import java.util.;<br>import java.net.;<br><br>public class BruteForce extends Thread<br>{<br> private static final String USERNAME = "";<br> private static final char [] POSSIBLE_CHAR =<br> {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm',<br> 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',<br> 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M',<br> 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'};<br> private static int NUMBER_OF_THREAD = 500;<br><br> private static Date startDate = null;<br> private static Date endDate = null;<br><br> private String address;<br> private String password;<br><br> public BruteForce(String address, String password)<br> {<br> this.address = address;<br> this.password = password;<br> }<br><br> public static void main(String[] args) throws IOException<br> {<br> if (args.length < 1)<br> {<br> System.err.println("Invalid usage!");<br> System.err.println("...</code>

• Loss: <code>SoftmaxLoss</code>

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

</details>

Training Logs

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 4.1.0
• Transformers: 4.52.4
• PyTorch: 2.6.0+cu124
• Accelerate: 1.7.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers and SoftmaxLoss

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->